-- Logs begin at Mon 2022-08-22 04:46:53 UTC, end at Mon 2022-08-22 06:05:40 UTC. -- Aug 22 05:24:11.043077 dv-cinder-853910-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 22 05:24:11.051214 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 22 05:24:11.053796 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 22 05:24:11 2022] *** Aug 22 05:24:11.053933 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 22 05:24:11.054045 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 22 05:24:11.054149 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: nodename: dv-cinder-853910-1-iscsi Aug 22 05:24:11.054243 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: machine: x86_64 Aug 22 05:24:11.054345 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: clock source: unix Aug 22 05:24:11.054444 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: pcre jit disabled Aug 22 05:24:11.054547 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: detected number of CPU cores: 4 Aug 22 05:24:11.054647 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: current working directory: / Aug 22 05:24:11.054745 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: detected binary path: /usr/bin/uwsgi-core Aug 22 05:24:11.054850 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: your processes number limit is 47680 Aug 22 05:24:11.054950 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: your memory page size is 4096 bytes Aug 22 05:24:11.055049 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: detected max file descriptor number: 2048 Aug 22 05:24:11.055156 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: lock engine: pthread robust mutexes Aug 22 05:24:11.055463 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: thunder lock: enabled Aug 22 05:24:11.056502 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 22 05:24:11.056629 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 22 05:24:11.096139 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: Python main interpreter initialized at 0x55826deb29c0 Aug 22 05:24:11.096139 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: python threads support enabled Aug 22 05:24:11.096139 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: your server socket listen backlog is limited to 100 connections Aug 22 05:24:11.096139 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: your mercy for graceful operations on workers is 90 seconds Aug 22 05:24:11.096469 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: mapped 403077 bytes (393 KB) for 2 cores Aug 22 05:24:11.096541 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: *** Operational MODE: preforking *** Aug 22 05:24:11.096800 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: *** uWSGI is running in multiple interpreter mode *** Aug 22 05:24:11.096800 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: spawned uWSGI master process (pid: 81574) Aug 22 05:24:11.097013 dv-cinder-853910-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 22 05:24:11.099242 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: spawned uWSGI worker 1 (pid: 81584, cores: 1) Aug 22 05:24:11.099242 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: spawned uWSGI worker 2 (pid: 81585, cores: 1) Aug 22 05:24:11.099242 dv-cinder-853910-1-iscsi devstack@keystone.service[81574]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 22 05:24:15.333956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 05:24:15.333956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: from cryptography.utils import int_from_bytes Aug 22 05:24:15.334738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 05:24:15.334738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: from cryptography.utils import int_from_bytes Aug 22 05:24:15.334930 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 05:24:15.334930 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: from cryptography.utils import int_from_bytes Aug 22 05:24:15.335368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 05:24:15.335368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: from cryptography.utils import int_from_bytes Aug 22 05:24:15.385360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 05:24:15.385605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 05:24:15.385679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 05:24:15.385749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 05:24:15.385808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] command line args: [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 05:24:15.385868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] command line args: [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 05:24:15.385935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 05:24:15.386130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 05:24:15.386210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ================================================================================ {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 05:24:15.386268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ================================================================================ {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 05:24:15.386399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] admin_token = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.386545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] admin_token = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.386620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] config_dir = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.386691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] config_dir = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.386750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.386910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] config_source = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] config_source = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] debug = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] debug = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.387943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] list_limit = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] list_limit = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] log_config_append = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] log_config_append = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.388989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] log_dir = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] log_dir = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] log_file = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] log_file = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.389937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390589 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.390825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391431 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.391997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.392150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.392277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.392424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.392553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.392648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.392720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] publish_errors = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] publish_errors = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.393966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] standard_threads = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] standard_threads = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.394954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] use_journal = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] use_journal = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] use_json = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] use_json = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.395898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] use_stderr = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.396015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] use_stderr = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.396100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] use_syslog = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.396171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] use_syslog = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.396238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.396311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 05:24:15.396380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.396447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.396598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.396720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.396807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.396879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.397880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398528 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.398866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.399912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.400911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.401860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.402976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.403958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.404994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.405942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.406834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.407965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.408992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.409906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.410991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.411970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.412942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.413972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.414943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.415109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.415286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.415427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.415530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.415668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.415791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.415869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416838 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.416992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.417934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.418935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.419878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.420841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] auth.external = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] auth.external = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.421992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] auth.password = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] auth.password = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] auth.token = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] auth.token = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.422918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.423976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] credential.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] credential.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.424908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.425859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.426995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.427073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.427263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.427340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.427515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.427605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.427787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.427863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.428988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429532 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.429927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.430968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.431957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.432971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.433912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.434863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.435892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.436960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.437986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.438928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.439894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440760 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.440995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441716 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.441947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.442973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.443991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.444228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.444304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.444462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.444540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.444712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.444789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.444960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445431 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.445952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.446935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.447934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.448954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.449901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.450847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.451976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.452933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.453966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] role.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.454901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] role.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] role.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] role.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.455878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.456853 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.457972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.458056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.458217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.458291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.458461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.458540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.458707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.458848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.459912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.460880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.461953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.462944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.463997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.464957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.465916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.466870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.467830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.connection = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.connection = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.468860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.469955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.470984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.471953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.472104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.472180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.472263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 05:24:15.472335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.472567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 05:24:15.473083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81585) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 05:24:15.502435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.502607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.502676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.502777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.502860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.503039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.511537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.511763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.544194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.544400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.545133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.545396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.598613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.598855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.648600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.648788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.648872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.648965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.810994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.811142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.811142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.811276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.834493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.834641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.834733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.834908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.834996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.835885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.847173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.847300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.860586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.860725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.860833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.860950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81585) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.861878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 05:24:15.878730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.878916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 05:24:15.879044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.879202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 05:24:15.879325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.879447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 05:24:15.879535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.879765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 05:24:15.879890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.880038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.880179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.880257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.880550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.880803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.880927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.881046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.890182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.890463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.890585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.890740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.890870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.891034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.891160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.891361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.898717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.899878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.900886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.901173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.901375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.901538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.901683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.901821 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.901958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.902094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.902231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.902391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.902528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.902663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.902747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.902877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.903031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.903121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.903282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.903480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.903720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.903869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.903979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.904278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.926117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.926853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.927560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.927873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.928339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.928630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.932583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.932871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.933156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.933497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.933786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.934071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.939433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.939752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.940429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.940697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.943813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.944106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.944394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.944678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.944800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.944955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.945047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.945197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.945344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.945478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.945631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.945750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.946005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.946313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.946611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.946886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.962854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.963135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.964240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.964525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.967002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.968401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.970221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.970521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.971614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.971940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.975039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.976490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.978314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.979830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.981565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.981868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.982153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.982434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.982715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.983002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.983125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.983359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.983667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.983956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.984237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.984517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.998738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:15.999024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.999307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:15.999606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.000373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.000660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.000953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.001231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.006428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.007937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.008068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.008227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.009648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.009941 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.011278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.011567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.011880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.012174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.012455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.012644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.012871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.013163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.013460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.013740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.023629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.023931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.024283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.024564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.024847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.025037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.025135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.025304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.025386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.025614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.025893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.026175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.026450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.026740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.040935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.042167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.043973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.045219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.046839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.048081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.048302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.049553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.049678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.049998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.050287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.051025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.051310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.051613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.056927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.058101 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.058372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.059549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.061340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.061641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.061928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.062202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.062524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.062823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.063119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.063409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.070105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.070399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.070683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.071396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.071719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.072007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.077566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81585) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.077848 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.078130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.078405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.078680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.078857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 05:24:16.078961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.079092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.079232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.079391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.079482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.079631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.079725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.079918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.080046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.080173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.080297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.080443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.080571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.080710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.080844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.081020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.081148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81585) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.081272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.081543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.081812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.082091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.082349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 05:24:16.115111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55826deb29c0 pid: 81585 (default app) Aug 22 05:24:16.116662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55826deb29c0 pid: 81584 (default app) Aug 22 05:24:16.118585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4967a-e747-4404-a21d-3655dbde9928 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:16.118870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4967a-e747-4404-a21d-3655dbde9928 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:16.119141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4967a-e747-4404-a21d-3655dbde9928 None None] PATH_INFO: `/v3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:16.123347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Mon Aug 22 05:24:16 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:22.745168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93ca3-f777-4af7-8b36-aca041c1072c None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:22.746114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93ca3-f777-4af7-8b36-aca041c1072c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:22.746114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93ca3-f777-4af7-8b36-aca041c1072c None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:22.753199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:22 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:22.762717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:22.763183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:22.763274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:22.841142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: INFO dbcounter [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] Registered counter for database keystone Aug 22 05:24:22.862713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG oslo_db.sqlalchemy.engines [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81585) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 05:24:22.874236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writer thread running {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 05:24:23.065678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG passlib.handlers.bcrypt [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81585) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 05:24:23.066010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG passlib.handlers.bcrypt [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81585) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 05:24:23.130149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:23.131789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:23.267692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 22 05:24:23.708364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93ca3-f777-4af7-8b36-aca041c1072c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:23.778774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: INFO dbcounter [None req-24c93ca3-f777-4af7-8b36-aca041c1072c None None] Registered counter for database keystone Aug 22 05:24:23.804029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG oslo_db.sqlalchemy.engines [None req-24c93ca3-f777-4af7-8b36-aca041c1072c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81584) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 05:24:23.812173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writer thread running {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 05:24:23.929987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93ca3-f777-4af7-8b36-aca041c1072c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:23.935354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a739917-5180-4f73-9f4f-112d0c1146f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:23.936028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739917-5180-4f73-9f4f-112d0c1146f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:23.936305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739917-5180-4f73-9f4f-112d0c1146f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:23.936560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739917-5180-4f73-9f4f-112d0c1146f4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:23.945777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a739917-5180-4f73-9f4f-112d0c1146f4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:23.998325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5a739917-5180-4f73-9f4f-112d0c1146f4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 05:24:23.998584 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a739917-5180-4f73-9f4f-112d0c1146f4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:24.008492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Mon Aug 22 05:24:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:24:24.015594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:24.033362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0eec-c153-4ec1-9625-e7e7717c6ac7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:24.040081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad16e4-9c6b-4856-bfdc-32562fae8bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:24.041370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad16e4-9c6b-4856-bfdc-32562fae8bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:24.041653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad16e4-9c6b-4856-bfdc-32562fae8bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:24.041918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad16e4-9c6b-4856-bfdc-32562fae8bc9 admin admin] PATH_INFO: `/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:24.043716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad16e4-9c6b-4856-bfdc-32562fae8bc9 admin admin] RBAC: Authorizing `identity:get_project(project_id=fb2b2de175ff44f7b8683cb1604be1f7)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:24.097083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f1ad16e4-9c6b-4856-bfdc-32562fae8bc9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 05:24:24.097423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad16e4-9c6b-4856-bfdc-32562fae8bc9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:24.102353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Mon Aug 22 05:24:24 2022] GET /identity/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:26.045516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ba5ff-e0d5-44ea-acf1-fbdd2d2fbbf1 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:26.046079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ba5ff-e0d5-44ea-acf1-fbdd2d2fbbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:26.046079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ba5ff-e0d5-44ea-acf1-fbdd2d2fbbf1 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:26.050201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:26.056372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e588c-dfce-4f78-822b-c0e9ef3b0599 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:26.056800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e588c-dfce-4f78-822b-c0e9ef3b0599 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:26.056888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e588c-dfce-4f78-822b-c0e9ef3b0599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:26.079875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bb0e588c-dfce-4f78-822b-c0e9ef3b0599 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:26.081438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bb0e588c-dfce-4f78-822b-c0e9ef3b0599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:26.091921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 05:24:26.356031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ba5ff-e0d5-44ea-acf1-fbdd2d2fbbf1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:26.370465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ba5ff-e0d5-44ea-acf1-fbdd2d2fbbf1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:26.375812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fce695-539e-48b5-a792-55f24fa1152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:26.376417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fce695-539e-48b5-a792-55f24fa1152b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:26.376671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fce695-539e-48b5-a792-55f24fa1152b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:26.376942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fce695-539e-48b5-a792-55f24fa1152b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:26.385887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fce695-539e-48b5-a792-55f24fa1152b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:26.387710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fce695-539e-48b5-a792-55f24fa1152b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:26.397009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Mon Aug 22 05:24:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:24:26.402703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e588c-dfce-4f78-822b-c0e9ef3b0599 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:26.412740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e588c-dfce-4f78-822b-c0e9ef3b0599 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:26.418420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72d37e-9fa0-4a04-878b-744b02ad2b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:26.419063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72d37e-9fa0-4a04-878b-744b02ad2b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:26.419340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72d37e-9fa0-4a04-878b-744b02ad2b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:26.419619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72d37e-9fa0-4a04-878b-744b02ad2b0f admin admin] PATH_INFO: `/v3/users/c35278cad6234455851a9ea018219e84` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:26.421143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72d37e-9fa0-4a04-878b-744b02ad2b0f admin admin] RBAC: Authorizing `identity:get_user(user_id=c35278cad6234455851a9ea018219e84)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:26.422994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72d37e-9fa0-4a04-878b-744b02ad2b0f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:26.424569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Mon Aug 22 05:24:26 2022] GET /identity/v3/users/c35278cad6234455851a9ea018219e84 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:28.374943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39979c76-6392-4fdd-be14-8dc4c97d6f1a None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:28.375480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39979c76-6392-4fdd-be14-8dc4c97d6f1a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:28.375480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39979c76-6392-4fdd-be14-8dc4c97d6f1a None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:28.376852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:28.382231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce851-3778-472a-900e-bb2bc12965c5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:28.382648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce851-3778-472a-900e-bb2bc12965c5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:28.382863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fce851-3778-472a-900e-bb2bc12965c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:28.401187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-24fce851-3778-472a-900e-bb2bc12965c5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:28.402368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-24fce851-3778-472a-900e-bb2bc12965c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:28.427363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165863f-8e7c-4c11-b640-31dd7d121338 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:28.427685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165863f-8e7c-4c11-b640-31dd7d121338 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:28.427962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165863f-8e7c-4c11-b640-31dd7d121338 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:28.429594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:28.435568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:28.435891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:28.436164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:28.479005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG passlib.handlers.bcrypt [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81584) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 05:24:28.479391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG passlib.handlers.bcrypt [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81584) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 05:24:28.487264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:28.524363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:28.525935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:28.626161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:28.749842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fce851-3778-472a-900e-bb2bc12965c5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:28.774875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:28.860695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fce851-3778-472a-900e-bb2bc12965c5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:28.885977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:28.895139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:28.895820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:28.896101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:28.896372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:28.912135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:28.920695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:28.921760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:28.922968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:28.927514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:28.993535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9e910-86e4-4d64-a71b-1a6e943e3430 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:28.995803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:29.013857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:29.030301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:29.031186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:29.031463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:29.031777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:29.032693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743a7cd-688d-4f59-bd05-b05518b60c97 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:29.066132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:29.066741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:29.067018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:29.067280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:29.067984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:29.068285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:29.069692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:29.081570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:24:29.087779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:29.102314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:29.103868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:29.188037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b488-15a4-419f-8559-e639d703f3c8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:29.201230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:29.206889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:29.210337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:29.222212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:29.223330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:29.223698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:29.223978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] PATH_INFO: `/v3/users/c35278cad6234455851a9ea018219e84` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:29.225176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] RBAC: Authorizing `identity:get_user(user_id=c35278cad6234455851a9ea018219e84)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:29.226655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:29.228708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:29 2022] GET /identity/v3/users/c35278cad6234455851a9ea018219e84 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:29.234476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:29.249237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:29.305863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a44f2c-b53e-45f2-89e4-223579ba5bc8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:29.335211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f329bd4-bbd6-4709-9da6-de272aab5eeb None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:29.338805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:29.339710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:29.339993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:29.340266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:29.359332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:29.360866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:29.373170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:29.373810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:29.374089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:29.374363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:29.375481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:29.376971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:29.378574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:24:29.379269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:29.380608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:24:29.381398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:24:29.381398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:24:29.381398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:29.381398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:29.384619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:29.386681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:29.445678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:29.469147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:29.490292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d397dd-abef-4c1b-b82d-81ae920c4d07 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:29.494476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9247f-4011-4b90-b753-1945d3762e49 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:29.524651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55139b6-a8b8-40e3-91f9-e180b3ae4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:29.525270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55139b6-a8b8-40e3-91f9-e180b3ae4988 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:29.525555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55139b6-a8b8-40e3-91f9-e180b3ae4988 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:29.525832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55139b6-a8b8-40e3-91f9-e180b3ae4988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:29.526575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55139b6-a8b8-40e3-91f9-e180b3ae4988 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.domain.id=default, user.id=c35278cad6234455851a9ea018219e84) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:29.526890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55139b6-a8b8-40e3-91f9-e180b3ae4988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:29.528283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358daed-da59-49d3-bbdf-60495ecdbe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:29.528404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55139b6-a8b8-40e3-91f9-e180b3ae4988 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:29.528897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358daed-da59-49d3-bbdf-60495ecdbe33 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:29.529176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358daed-da59-49d3-bbdf-60495ecdbe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:29.529446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358daed-da59-49d3-bbdf-60495ecdbe33 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:29.530156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358daed-da59-49d3-bbdf-60495ecdbe33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:29.530385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358daed-da59-49d3-bbdf-60495ecdbe33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:29.531790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358daed-da59-49d3-bbdf-60495ecdbe33 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:29.535006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Mon Aug 22 05:24:29 2022] GET /identity/v3/role_assignments?user.id=c35278cad6234455851a9ea018219e84&scope.domain.id=default&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:29.551187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:24:31.546154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0a7b-00b7-4c64-a3c4-e4690f1c0ced None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:31.546693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0a7b-00b7-4c64-a3c4-e4690f1c0ced None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:31.546693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0a7b-00b7-4c64-a3c4-e4690f1c0ced None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:31.548366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:31.553516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71392f0e-fbc8-4c63-8885-93094d83034c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:31.553901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71392f0e-fbc8-4c63-8885-93094d83034c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:31.554060 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71392f0e-fbc8-4c63-8885-93094d83034c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:31.580705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-71392f0e-fbc8-4c63-8885-93094d83034c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:31.581897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-71392f0e-fbc8-4c63-8885-93094d83034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:31.681708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:31.754260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147687a-a2c9-4caf-b4e0-033423eb204a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:31.754599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147687a-a2c9-4caf-b4e0-033423eb204a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:31.754810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147687a-a2c9-4caf-b4e0-033423eb204a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:31.756203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:31.767171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cda9e4-f254-4632-87d7-5363a79068c0 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:31.767461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cda9e4-f254-4632-87d7-5363a79068c0 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:31.767557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808277f-c5c1-4324-b77e-31f761aedd05 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:31.767762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cda9e4-f254-4632-87d7-5363a79068c0 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:31.767888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808277f-c5c1-4324-b77e-31f761aedd05 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:31.768051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808277f-c5c1-4324-b77e-31f761aedd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:31.769201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 12/25] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:31.778824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92eea7-8f72-42d8-9612-d1d2e397c0d9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:31.779169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92eea7-8f72-42d8-9612-d1d2e397c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:31.779386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92eea7-8f72-42d8-9612-d1d2e397c0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:31.795048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4808277f-c5c1-4324-b77e-31f761aedd05 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:31.796365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4808277f-c5c1-4324-b77e-31f761aedd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:31.812615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6e92eea7-8f72-42d8-9612-d1d2e397c0d9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:31.813840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6e92eea7-8f72-42d8-9612-d1d2e397c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:31.870258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 14/26] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:31.907287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:31.973286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 13/27] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:32.189969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808277f-c5c1-4324-b77e-31f761aedd05 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:32.196299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92eea7-8f72-42d8-9612-d1d2e397c0d9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:32.314595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:32.322183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808277f-c5c1-4324-b77e-31f761aedd05 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:32.388265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:32.389107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:32.389394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:32.389668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:32.390408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:32.392073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:32.462072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92eea7-8f72-42d8-9612-d1d2e397c0d9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:32.510252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0566e5b-87f8-4ed0-89df-a6aff7b2404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:32.510896 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0566e5b-87f8-4ed0-89df-a6aff7b2404e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:32.511172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0566e5b-87f8-4ed0-89df-a6aff7b2404e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:32.511440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0566e5b-87f8-4ed0-89df-a6aff7b2404e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:32.522601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0566e5b-87f8-4ed0-89df-a6aff7b2404e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b0566e5b-87f8-4ed0-89df-a6aff7b2404e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:32.529144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:32.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:32.530726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:32.535084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 14/28] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:24:32.536342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f5a8-cf6b-4706-82ef-c8d835f510d9 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:32.536628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f5a8-cf6b-4706-82ef-c8d835f510d9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:32.536899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f5a8-cf6b-4706-82ef-c8d835f510d9 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:32.542416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 15/29] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:32.543241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f5a8-cf6b-4706-82ef-c8d835f510d9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:32.549488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 15/30] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 22 05:24:32 2022] POST /identity/v3/roles => generated 219 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:32.550336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:32.603393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:32.717072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f5a8-cf6b-4706-82ef-c8d835f510d9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:32.751420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff517d-aeb8-430b-a97f-8c1ed13c2ff6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:32.770475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf6b9-55e3-400b-9cbd-e245547cd950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:32.771518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf6b9-55e3-400b-9cbd-e245547cd950 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:32.772053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf6b9-55e3-400b-9cbd-e245547cd950 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:32.772882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf6b9-55e3-400b-9cbd-e245547cd950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:32.788521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf6b9-55e3-400b-9cbd-e245547cd950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:32.790556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf6b9-55e3-400b-9cbd-e245547cd950 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:32.804073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:32.804883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:32.805405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:32.806156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:32.806913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:32.807223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:32.810833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-cc7bf6b9-55e3-400b-9cbd-e245547cd950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:24:32.811191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:32.812095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:24:32.812909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:24:32.812909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:24:32.812909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:32.812909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:32.813929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:32.818227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8315a-b5f4-4bb8-8b1f-0e50189d3edf None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:32.818506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8315a-b5f4-4bb8-8b1f-0e50189d3edf None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:32.818779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8315a-b5f4-4bb8-8b1f-0e50189d3edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:32.821565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 16/32] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:32.827144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:32.839160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-dbc8315a-b5f4-4bb8-8b1f-0e50189d3edf None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:32.840642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-dbc8315a-b5f4-4bb8-8b1f-0e50189d3edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:32.928039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 17/33] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:32.930076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8315a-b5f4-4bb8-8b1f-0e50189d3edf None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:32.976040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:32.977844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11e6fc-93ad-49e6-aa16-832d8ef5c129 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:33.047498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:33.049130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.049773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.051267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.052857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:33.053889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:33.056261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:33.103449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 17/34] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:33.110582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:33.135334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8315a-b5f4-4bb8-8b1f-0e50189d3edf None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:33.189643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:33.190672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.191135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.191643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] PATH_INFO: `/v3/users/c35278cad6234455851a9ea018219e84` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.193424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] RBAC: Authorizing `identity:get_user(user_id=c35278cad6234455851a9ea018219e84)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:33.195813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:33.198208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 18/35] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:32 2022] GET /identity/v3/users/c35278cad6234455851a9ea018219e84 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:33.203900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:33.228102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:33.279750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9212-291c-4324-8d84-6b21be811951 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:33.322779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:33.330535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:33.331975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.332476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.333060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.334186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:33.336689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:33.340807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb29e9-e0e1-4213-8453-effa94ed87c2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:33.387719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 18/36] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:24:33 2022] POST /identity/v3/projects => generated 301 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:33.387719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:33.394225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f0c5e-9502-454c-b9ac-009fce3b0a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:33.395081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f0c5e-9502-454c-b9ac-009fce3b0a08 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.395794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f0c5e-9502-454c-b9ac-009fce3b0a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.396322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f0c5e-9502-454c-b9ac-009fce3b0a08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.398022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f0c5e-9502-454c-b9ac-009fce3b0a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:33.400204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f0c5e-9502-454c-b9ac-009fce3b0a08 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:33.402170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 19/37] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:33.405112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b967d1-a1ef-4660-92a9-d1848a8df6e6 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.405396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b967d1-a1ef-4660-92a9-d1848a8df6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.405781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b967d1-a1ef-4660-92a9-d1848a8df6e6 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.407619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 20/38] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:33.414852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca543d-4bfb-47b9-958e-2cdea326c641 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.415604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca543d-4bfb-47b9-958e-2cdea326c641 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.416090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca543d-4bfb-47b9-958e-2cdea326c641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.440991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-beca543d-4bfb-47b9-958e-2cdea326c641 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:33.442595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-beca543d-4bfb-47b9-958e-2cdea326c641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:33.533425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37b26b-210a-4843-9e66-f9e28c7b6c5d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:33.546364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:33.548927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 21/39] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:33.553871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca543d-4bfb-47b9-958e-2cdea326c641 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:33.593629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677dc131-783e-4d63-ab86-59644b2c5ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:33.594314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dc131-783e-4d63-ab86-59644b2c5ee7 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.594540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dc131-783e-4d63-ab86-59644b2c5ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.595155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dc131-783e-4d63-ab86-59644b2c5ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.596025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677dc131-783e-4d63-ab86-59644b2c5ee7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:33.597573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677dc131-783e-4d63-ab86-59644b2c5ee7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:33.620457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 19/40] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 22 05:24:33 2022] POST /identity/v3/roles => generated 225 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:33.687621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:33.711277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca543d-4bfb-47b9-958e-2cdea326c641 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:33.729927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffe35-d4a7-4e3b-831b-0e42fd78270f None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.730570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffe35-d4a7-4e3b-831b-0e42fd78270f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.731031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffe35-d4a7-4e3b-831b-0e42fd78270f None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.734541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 20/41] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:33.741302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc08a7-4073-461b-be17-d0363ffdfce2 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.741904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc08a7-4073-461b-be17-d0363ffdfce2 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.742583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc08a7-4073-461b-be17-d0363ffdfce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.752733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:33.753485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:33.753761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:33.754033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] PATH_INFO: `/v3/domains/default/users/c35278cad6234455851a9ea018219e84/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:33.756527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c35278cad6234455851a9ea018219e84, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:33.759879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:33.776297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-babc08a7-4073-461b-be17-d0363ffdfce2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:33.777971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-babc08a7-4073-461b-be17-d0363ffdfce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:33.862963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 21/42] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:33.894901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc08a7-4073-461b-be17-d0363ffdfce2 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:33.905763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 22/43] 10.222.0.18 () {58 vars in 1361 bytes} [Mon Aug 22 05:24:33 2022] PUT /identity/v3/domains/default/users/c35278cad6234455851a9ea018219e84/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:24:34.021157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:34.025903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc08a7-4073-461b-be17-d0363ffdfce2 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:34.077228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:34.078274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:34.078541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:34.078811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:34.079493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:34.080959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:34.100709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 22/44] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 22 05:24:33 2022] POST /identity/v3/roles => generated 223 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:34.154652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:34.292092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d21242-efa4-47fa-b532-b4c5fdc46b11 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:34.333079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f96c4-0ce5-4b5e-b26d-012702fcca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:34.333745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f96c4-0ce5-4b5e-b26d-012702fcca72 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:34.334139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f96c4-0ce5-4b5e-b26d-012702fcca72 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:34.334418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f96c4-0ce5-4b5e-b26d-012702fcca72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:34.335700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f96c4-0ce5-4b5e-b26d-012702fcca72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:34.337924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f96c4-0ce5-4b5e-b26d-012702fcca72 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:34.342404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 23/45] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:34.348673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:34.462063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e23889-5a9f-4686-b33d-2efc7b54aa4d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:34.498019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6420c7-0eb7-4bf1-a6d6-88867e51e265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:34.498651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6420c7-0eb7-4bf1-a6d6-88867e51e265 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:34.498932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6420c7-0eb7-4bf1-a6d6-88867e51e265 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:34.499199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6420c7-0eb7-4bf1-a6d6-88867e51e265 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:34.499888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6420c7-0eb7-4bf1-a6d6-88867e51e265 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:34.501366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6420c7-0eb7-4bf1-a6d6-88867e51e265 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:34.529975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 23/46] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:24:34 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:24:35.887410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6fa2-3946-4a21-8078-0ccdd7de626e None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:35.887953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6fa2-3946-4a21-8078-0ccdd7de626e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:35.888032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6fa2-3946-4a21-8078-0ccdd7de626e None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:35.889344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 24/47] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:35.894649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100446-aaed-4220-bd76-3c15f78d02d6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:35.895129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100446-aaed-4220-bd76-3c15f78d02d6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:35.895203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100446-aaed-4220-bd76-3c15f78d02d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:36.025726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-31100446-aaed-4220-bd76-3c15f78d02d6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:36.027293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-31100446-aaed-4220-bd76-3c15f78d02d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:36.030482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:36.099667 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 24/48] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:36.361555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce6fa2-3946-4a21-8078-0ccdd7de626e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:36.378194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:36.419287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c53b0-a200-452c-b9d2-02f399f203ed None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:36.419557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c53b0-a200-452c-b9d2-02f399f203ed None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:36.419868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c53b0-a200-452c-b9d2-02f399f203ed None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:36.421226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 25/49] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:36.426369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9f8a-706e-4b27-baf0-34f2877905b6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:36.426657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9f8a-706e-4b27-baf0-34f2877905b6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:36.426956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9f8a-706e-4b27-baf0-34f2877905b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:36.445335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8efd9f8a-706e-4b27-baf0-34f2877905b6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:36.447367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8efd9f8a-706e-4b27-baf0-34f2877905b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:36.470243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce6fa2-3946-4a21-8078-0ccdd7de626e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:36.508919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:36.509543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:36.509826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:36.510096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:36.521521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:36.522632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 26/50] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:36.526323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:36.527298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:36.528102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 25/51] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:36.534656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9f8a-706e-4b27-baf0-34f2877905b6 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:36.604225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:36.637728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9f8a-706e-4b27-baf0-34f2877905b6 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:36.673149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:36.673767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:36.674049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:36.674320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:36.675000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:36.675308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:36.676760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:36.685547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 27/52] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:36.691269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:36.784241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:36.875037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:36.891463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7f4a8-0b1c-404f-aa38-2d40855ef718 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:36.908857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbf067-6d36-44d9-92bb-e5f267c2f58c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:36.926317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:36.932221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:36.932856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:36.933139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:36.933443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:36.934604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:36.936120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:36.937707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 28/53] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:36.943175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:36.943780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:36.944432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:36.944714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:36.944990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] PATH_INFO: `/v3/users/c35278cad6234455851a9ea018219e84` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:36.946292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] RBAC: Authorizing `identity:get_user(user_id=c35278cad6234455851a9ea018219e84)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:36.947925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:36.949529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:36 2022] GET /identity/v3/users/c35278cad6234455851a9ea018219e84 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:36.954900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:37.041679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d2820-4b5e-4cb4-904b-3c4fbe87a560 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:37.057226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1c83f-e8a3-467a-9b4a-ab78d3b75e80 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:37.075089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e38b57-5242-47c0-af3e-7fb50c4aa3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:37.075748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38b57-5242-47c0-af3e-7fb50c4aa3b4 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:37.076035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38b57-5242-47c0-af3e-7fb50c4aa3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:37.076308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38b57-5242-47c0-af3e-7fb50c4aa3b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:37.076956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e38b57-5242-47c0-af3e-7fb50c4aa3b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:37.078437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e38b57-5242-47c0-af3e-7fb50c4aa3b4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:37.089666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:37.090325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:37.090609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:37.090875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:37.092023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:37.093485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:37.095083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 27/55] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:24:37.101282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:37.110042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 29/56] 10.222.0.18 () {60 vars in 1140 bytes} [Mon Aug 22 05:24:36 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:37.112544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:37.210088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f526-f94d-4c37-a588-0bf1f472b773 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:37.247460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6633cb-55b0-42d5-8eda-760daa750c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:37.248123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6633cb-55b0-42d5-8eda-760daa750c70 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:37.248414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6633cb-55b0-42d5-8eda-760daa750c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:37.248671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6633cb-55b0-42d5-8eda-760daa750c70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:37.249426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6633cb-55b0-42d5-8eda-760daa750c70 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.domain.id=default, user.id=c35278cad6234455851a9ea018219e84) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:37.249858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6633cb-55b0-42d5-8eda-760daa750c70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:37.251558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6633cb-55b0-42d5-8eda-760daa750c70 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:37.261015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 28/57] 10.222.0.18 () {56 vars in 1322 bytes} [Mon Aug 22 05:24:37 2022] GET /identity/v3/role_assignments?user.id=c35278cad6234455851a9ea018219e84&scope.domain.id=default&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:24:39.007079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdfb40-5e7e-4885-8c7a-e336eed8fcb3 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:39.007967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdfb40-5e7e-4885-8c7a-e336eed8fcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:39.007967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdfb40-5e7e-4885-8c7a-e336eed8fcb3 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:39.010186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 30/58] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:39.016595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceec27-234e-4548-937d-cd6dab9d801e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:39.017261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceec27-234e-4548-937d-cd6dab9d801e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:39.017530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceec27-234e-4548-937d-cd6dab9d801e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:39.040336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3cceec27-234e-4548-937d-cd6dab9d801e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:39.041890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3cceec27-234e-4548-937d-cd6dab9d801e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:39.114684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 29/59] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:39.388461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdfb40-5e7e-4885-8c7a-e336eed8fcb3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:39.459836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:39.527762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdfb40-5e7e-4885-8c7a-e336eed8fcb3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:39.575181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691ceed-b376-4996-a5c6-3548375d7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:39.575844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691ceed-b376-4996-a5c6-3548375d7938 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:39.576123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691ceed-b376-4996-a5c6-3548375d7938 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:39.576398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691ceed-b376-4996-a5c6-3548375d7938 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:39.577588 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691ceed-b376-4996-a5c6-3548375d7938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:39.579061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691ceed-b376-4996-a5c6-3548375d7938 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:39.580676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 31/60] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:39.586926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceec27-234e-4548-937d-cd6dab9d801e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:39.589264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:39.702444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceec27-234e-4548-937d-cd6dab9d801e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:39.751951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375551f5-b70e-47ac-89fd-5e6b52e2ae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:39.752575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375551f5-b70e-47ac-89fd-5e6b52e2ae35 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:39.752857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375551f5-b70e-47ac-89fd-5e6b52e2ae35 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:39.753129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375551f5-b70e-47ac-89fd-5e6b52e2ae35 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:39.753778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375551f5-b70e-47ac-89fd-5e6b52e2ae35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:39.755231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375551f5-b70e-47ac-89fd-5e6b52e2ae35 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:39.785521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 30/61] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:24:39 2022] POST /identity/v3/users => generated 275 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:42.771697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f710-8d9f-49a1-a3ee-43e97ca7659b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:42.772216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f710-8d9f-49a1-a3ee-43e97ca7659b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:42.772288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f710-8d9f-49a1-a3ee-43e97ca7659b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:42.773738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 32/62] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:42.787870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3f5e7-b2b5-4c4b-ab30-9024f1bb6288 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:42.788817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3f5e7-b2b5-4c4b-ab30-9024f1bb6288 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:42.789298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3f5e7-b2b5-4c4b-ab30-9024f1bb6288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:42.812325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-69a3f5e7-b2b5-4c4b-ab30-9024f1bb6288 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:42.813991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-69a3f5e7-b2b5-4c4b-ab30-9024f1bb6288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:42.853581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730c3e8-e511-406c-bf4f-9ae148dfdd61 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:42.854230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730c3e8-e511-406c-bf4f-9ae148dfdd61 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:42.854713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730c3e8-e511-406c-bf4f-9ae148dfdd61 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:42.856309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 33/63] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:42.861935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5eae-d440-44f5-8b37-5ed430b3f835 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:42.862450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5eae-d440-44f5-8b37-5ed430b3f835 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:42.862917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5eae-d440-44f5-8b37-5ed430b3f835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:42.881188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d54e5eae-d440-44f5-8b37-5ed430b3f835 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:42.882727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d54e5eae-d440-44f5-8b37-5ed430b3f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:42.889178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:42.945529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 31/64] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:42.954227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 34/65] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:43.203025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b96f76-8d2a-4754-bb1f-dd2e10e5f89f None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.203315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b96f76-8d2a-4754-bb1f-dd2e10e5f89f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.203621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b96f76-8d2a-4754-bb1f-dd2e10e5f89f None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.205005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 32/66] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:43.210432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b8b4-c737-4536-90b0-35539ae2729d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.210929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b8b4-c737-4536-90b0-35539ae2729d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.211371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b8b4-c737-4536-90b0-35539ae2729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.219338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b96f76-8d2a-4754-bb1f-dd2e10e5f89f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:43.241640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7b90b8b4-c737-4536-90b0-35539ae2729d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:43.243266 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7b90b8b4-c737-4536-90b0-35539ae2729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:43.333484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:43.361944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b96f76-8d2a-4754-bb1f-dd2e10e5f89f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:43.366263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 35/67] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:43.367026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90b8b4-c737-4536-90b0-35539ae2729d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:43.426966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405dd7a-779d-4eed-9e91-b09f18c82509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:43.427756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405dd7a-779d-4eed-9e91-b09f18c82509 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.428039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405dd7a-779d-4eed-9e91-b09f18c82509 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.428309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405dd7a-779d-4eed-9e91-b09f18c82509 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.439116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405dd7a-779d-4eed-9e91-b09f18c82509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-6405dd7a-779d-4eed-9e91-b09f18c82509 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:43.444144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:43.445071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:43.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:43.450261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 33/68] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:43.453338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e42e9-cd54-46dc-b0d7-2fadce21514f None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.453572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e42e9-cd54-46dc-b0d7-2fadce21514f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.453904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e42e9-cd54-46dc-b0d7-2fadce21514f None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.455289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 34/69] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:43.456580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e42e9-cd54-46dc-b0d7-2fadce21514f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:43.510221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90b8b4-c737-4536-90b0-35539ae2729d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:43.511352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:43.550024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cfbf0-6dd1-4a37-b0a3-2321d839680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:43.551021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfbf0-6dd1-4a37-b0a3-2321d839680e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.551489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfbf0-6dd1-4a37-b0a3-2321d839680e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.552063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfbf0-6dd1-4a37-b0a3-2321d839680e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.565039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cfbf0-6dd1-4a37-b0a3-2321d839680e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:43.565367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e42e9-cd54-46dc-b0d7-2fadce21514f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-000cfbf0-6dd1-4a37-b0a3-2321d839680e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:43.570824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:43.571679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:43.572461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:43.574877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 36/70] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:24:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:43.577003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de495d4-bd91-447d-a4ba-17d56d8b6d84 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.577493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de495d4-bd91-447d-a4ba-17d56d8b6d84 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.577934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de495d4-bd91-447d-a4ba-17d56d8b6d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.595418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7de495d4-bd91-447d-a4ba-17d56d8b6d84 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:43.596637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7de495d4-bd91-447d-a4ba-17d56d8b6d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:43.601664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a940ce1-0124-44fc-b6ae-8dbcad40893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:43.602294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940ce1-0124-44fc-b6ae-8dbcad40893a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.602576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940ce1-0124-44fc-b6ae-8dbcad40893a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.602850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940ce1-0124-44fc-b6ae-8dbcad40893a None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.603673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940ce1-0124-44fc-b6ae-8dbcad40893a None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:43.603984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940ce1-0124-44fc-b6ae-8dbcad40893a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:43.605520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940ce1-0124-44fc-b6ae-8dbcad40893a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:43.619960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 35/71] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:43.621744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c869746-518d-4e53-9845-948a479560ff None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.622374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c869746-518d-4e53-9845-948a479560ff None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.623464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c869746-518d-4e53-9845-948a479560ff None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.625308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 36/72] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:43.625886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c869746-518d-4e53-9845-948a479560ff None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:43.638521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:43.672175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 37/73] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:43.673026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de495d4-bd91-447d-a4ba-17d56d8b6d84 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:43.731600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c869746-518d-4e53-9845-948a479560ff None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:43.743604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:43.772265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f23f9a-830b-4307-b374-3d91d534efb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:43.772892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f23f9a-830b-4307-b374-3d91d534efb4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.773167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f23f9a-830b-4307-b374-3d91d534efb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.773438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f23f9a-830b-4307-b374-3d91d534efb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.774106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f23f9a-830b-4307-b374-3d91d534efb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:43.774409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f23f9a-830b-4307-b374-3d91d534efb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:43.776414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f23f9a-830b-4307-b374-3d91d534efb4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:43.777550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de495d4-bd91-447d-a4ba-17d56d8b6d84 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:43.785246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 37/74] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:43.790995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95add7a8-e5d4-4e81-9977-e105a9c759b2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.791641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95add7a8-e5d4-4e81-9977-e105a9c759b2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.791641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95add7a8-e5d4-4e81-9977-e105a9c759b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.808821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-95add7a8-e5d4-4e81-9977-e105a9c759b2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:43.810074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-95add7a8-e5d4-4e81-9977-e105a9c759b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:43.818333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:43.819158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.819598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.819884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:43.835527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:43.837121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:43.839431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 38/75] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:43 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:43.840110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:43.868995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:43.887625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 38/76] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:43.887625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95add7a8-e5d4-4e81-9977-e105a9c759b2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:43.958824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56318-4caf-487c-bc5c-9b78e0ef0fa9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:43.993161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95add7a8-e5d4-4e81-9977-e105a9c759b2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:43.994896 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:43.995869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:43.996541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:43.996796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:43.997063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.008272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:44.013278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:44.014094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:44.014858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 39/77] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:24:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:44.016381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.032536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.033173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.033473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.033773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] PATH_INFO: `/v3/users/c35278cad6234455851a9ea018219e84` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.035166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] RBAC: Authorizing `identity:get_user(user_id=c35278cad6234455851a9ea018219e84)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.036673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.038285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 39/78] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:43 2022] GET /identity/v3/users/c35278cad6234455851a9ea018219e84 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:44.041079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.108672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.121630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57849b-4fde-4fff-a2a0-35f75a41c5b1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.146159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605f3a2-05e6-4506-b058-cfaec382ceb5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.157915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.158529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.158808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.159079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.179890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.180530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.180811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.181078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.191163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:44.195964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:44.196929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:44.197887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 40/79] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:44.199890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.226255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.227830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.229840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.297359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd8e3-82f0-4c0a-bcd7-b66a17219861 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.314874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.328669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 40/80] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:44.329485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.335326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.335985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.336260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.336525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.337196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:44.337563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.339461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.348093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 41/81] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:44.348871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.430150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113b538-fb11-48e6-8b43-1612c1e589ee None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.437880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.447285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4ebf-b173-4050-9806-5dd085a98353 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.463955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.464574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.464850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.465118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.480697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.481336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.481615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.481885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.482983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.484475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.486063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 42/82] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:44.486816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.522519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.534991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.536523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.584614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe04f60-19bc-4270-805e-afaf47cb7b7e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.620426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.621050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.621336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.621602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.622267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:44.622571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.624012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.632262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 43/83] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:44.633038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.643600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 41/84] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:44.644456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.666675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.732986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.741207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797f207-3c2c-4b55-86de-cbbb7e9f04ce None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.745042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690a47b-fc93-4077-b34c-bd291d722984 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.774640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.775319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.775605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.775872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.776971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.778367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.778833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.779129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.779473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.779888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.780115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 44/85] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:44.785449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.785527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.785527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.785527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 42/86] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:44.785527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.876079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.882105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfea8bc-11f0-4c01-b7f0-1612c533ea32 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.886447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a791b-6943-4718-9bac-849d91d33b58 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:44.916533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.917183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.917437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.917727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.918951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.920469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.926709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 43/87] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:44.926709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:44.926904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:44.926904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:44.926904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:44.927167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:44.927823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:44.929278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:44.949975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:44.950793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 45/88] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:24:44 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:44.951637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.040130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146e056-fa5f-4cb1-be44-24b0b22b448b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:45.077809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebffe3-f865-49bc-a63d-8a0f181c7b16 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:45.079143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:45.079928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:45.080209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:45.080482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:45.081702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:45.083149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:45.089884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 44/89] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:45.089884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.129249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:45.129906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:45.130975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:45.130975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:45.132215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=f27b32aa767849b1b542bbb8f3eeeb4e) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:45.132535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:45.134164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:45.141553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 46/90] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:44 2022] GET /identity/v3/role_assignments?user.id=f27b32aa767849b1b542bbb8f3eeeb4e&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=69140820db3a4dfea9428bdde1203a39 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:45.144371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.160853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:45.218457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:45.236470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588fa24-5886-4c5a-8687-79bd4b8ca483 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:45.286296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:45.286945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:45.287226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:45.287497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:45.288136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7585a87-64ee-4e03-afdd-98ca28c50a60 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:45.329042 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:45.329952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:45.330518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:45.331079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:45.375988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:45.377512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:45.418887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:45.420329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:45.434991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:45.492422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:45.503982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 45/91] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:45.504976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.554738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 47/92] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:45.556372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.654514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c631dc-2601-4797-99df-a5296929b130 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:45.679907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1d0df-5502-4f3e-b4c4-ce81bf21a4c4 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:45.693539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:45.724793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:45.727228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:45.727755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:45.728593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:45.730437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:45.731258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:45.732087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:45.733755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:45.734239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:45.734715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:45.735679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=c35278cad6234455851a9ea018219e84) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:45.737551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 48/93] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:45 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:45.738142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:45.739880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:45.740841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.758583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 46/94] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:45 2022] GET /identity/v3/role_assignments?user.id=c35278cad6234455851a9ea018219e84&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:45.759248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.863957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cce40-3e17-4c41-bf36-eb60fed78a8c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:45.916898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:45.917970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:45.920708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:45.922390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:45.922867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] PATH_INFO: `/v3/projects/f7e9106180e64d35883fab40097dbb31` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:45.926177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] RBAC: Authorizing `identity:get_project(project_id=f7e9106180e64d35883fab40097dbb31)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:45.928711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:45.934874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 49/95] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:45 2022] GET /identity/v3/projects/f7e9106180e64d35883fab40097dbb31 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:45.940949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:45.991034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434faa5-1d61-444e-a683-14317fb9503a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:46.039857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850e2d1-0b5b-4b30-bbd5-241073ba1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:46.040925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850e2d1-0b5b-4b30-bbd5-241073ba1f91 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:46.041397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850e2d1-0b5b-4b30-bbd5-241073ba1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:46.041878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850e2d1-0b5b-4b30-bbd5-241073ba1f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:46.042803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850e2d1-0b5b-4b30-bbd5-241073ba1f91 None admin] RBAC: Adding query filter params (role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=f27b32aa767849b1b542bbb8f3eeeb4e) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:46.043834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850e2d1-0b5b-4b30-bbd5-241073ba1f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:46.045912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850e2d1-0b5b-4b30-bbd5-241073ba1f91 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:46.052397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 47/96] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:45 2022] GET /identity/v3/role_assignments?user.id=f27b32aa767849b1b542bbb8f3eeeb4e&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:46.062005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:46.152711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ab0af-b216-46c5-a367-d917c16a0f8a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:46.204951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4b11e-bbcd-4231-8774-3b9a66829be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:46.205563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b11e-bbcd-4231-8774-3b9a66829be8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:46.205848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b11e-bbcd-4231-8774-3b9a66829be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:46.206124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b11e-bbcd-4231-8774-3b9a66829be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:46.206979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4b11e-bbcd-4231-8774-3b9a66829be8 None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=f7e9106180e64d35883fab40097dbb31, user.id=f27b32aa767849b1b542bbb8f3eeeb4e) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:46.207296 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4b11e-bbcd-4231-8774-3b9a66829be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:46.208784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4b11e-bbcd-4231-8774-3b9a66829be8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:46.214948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 50/97] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:45 2022] GET /identity/v3/role_assignments?user.id=f27b32aa767849b1b542bbb8f3eeeb4e&scope.project.id=f7e9106180e64d35883fab40097dbb31&role.id=69140820db3a4dfea9428bdde1203a39 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:47.771088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3425c29-b080-48d4-a9fb-f19b3de843cf None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:47.771740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3425c29-b080-48d4-a9fb-f19b3de843cf None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:47.771740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3425c29-b080-48d4-a9fb-f19b3de843cf None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:47.773158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 48/98] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:47.785519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3994-e9f8-4f0c-b487-b4a12dc335ea None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:47.786025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3994-e9f8-4f0c-b487-b4a12dc335ea None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:47.786231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3994-e9f8-4f0c-b487-b4a12dc335ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:47.816586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-200e3994-e9f8-4f0c-b487-b4a12dc335ea None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:47.817785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-200e3994-e9f8-4f0c-b487-b4a12dc335ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:47.903317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 51/99] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:47.929624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2aef-62a8-400e-96fc-43ed2fec5624 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:47.930237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2aef-62a8-400e-96fc-43ed2fec5624 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:47.930711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2aef-62a8-400e-96fc-43ed2fec5624 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:47.932506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 49/100] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:47.938189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2edf3e-b8c3-4902-80da-e683ca97bc17 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:47.938741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2edf3e-b8c3-4902-80da-e683ca97bc17 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:47.939235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2edf3e-b8c3-4902-80da-e683ca97bc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:47.964870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9b2edf3e-b8c3-4902-80da-e683ca97bc17 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:47.966468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9b2edf3e-b8c3-4902-80da-e683ca97bc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:47.982305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b796f-3086-44c2-b099-7a22e51ec957 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:47.982897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b796f-3086-44c2-b099-7a22e51ec957 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:47.983390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b796f-3086-44c2-b099-7a22e51ec957 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:47.984967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 50/101] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:47.998700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953242-d96f-4e1d-847d-89813d793a0b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:47.999250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953242-d96f-4e1d-847d-89813d793a0b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:47.999773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953242-d96f-4e1d-847d-89813d793a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:48.021350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c0953242-d96f-4e1d-847d-89813d793a0b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:48.022892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c0953242-d96f-4e1d-847d-89813d793a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:48.054201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:48.145132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 51/102] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 22 05:24:48.155434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 52/103] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:48.497285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953242-d96f-4e1d-847d-89813d793a0b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:48.533398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2edf3e-b8c3-4902-80da-e683ca97bc17 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:48.570671 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:48.631807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953242-d96f-4e1d-847d-89813d793a0b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:48.675658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:48.677089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:48.677517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:48.677933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:48.679695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:48.682030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:48.684795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 52/104] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:48.685808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:48.709019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2edf3e-b8c3-4902-80da-e683ca97bc17 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:48.758798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a31bf6-d653-4171-ab71-bb12b7882527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:48.759640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a31bf6-d653-4171-ab71-bb12b7882527 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:48.760017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a31bf6-d653-4171-ab71-bb12b7882527 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:48.760380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a31bf6-d653-4171-ab71-bb12b7882527 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:48.774509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a31bf6-d653-4171-ab71-bb12b7882527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-97a31bf6-d653-4171-ab71-bb12b7882527 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:48.780723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:48.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:48.782494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 53/105] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:48.784490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd3e3-d05e-4067-91f1-8163c795b12c None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:48.786224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd3e3-d05e-4067-91f1-8163c795b12c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:48.786769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd3e3-d05e-4067-91f1-8163c795b12c None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:48.788041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 54/106] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:48.792342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cd3e3-d05e-4067-91f1-8163c795b12c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:48.819781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71bf6-7bd6-4c35-8f8a-faf04dde2697 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:48.828063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:48.837945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:48.855186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:48.855942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:48.856135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:48.856402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:48.867656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:48.872541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:48.873400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:48.874153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 53/107] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:48.874768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:48.919634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cd3e3-d05e-4067-91f1-8163c795b12c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:48.955136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73a231-42cd-45fa-901e-3884a8d92952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:48.955786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73a231-42cd-45fa-901e-3884a8d92952 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:48.956071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73a231-42cd-45fa-901e-3884a8d92952 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:48.956345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73a231-42cd-45fa-901e-3884a8d92952 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:48.956993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73a231-42cd-45fa-901e-3884a8d92952 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:48.958569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73a231-42cd-45fa-901e-3884a8d92952 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:48.984920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598157d8-83e2-4285-8fc2-f6843c4fbda6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:48.997592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 55/108] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:24:48 2022] POST /identity/v3/projects => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:48.997592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34dd20-2053-4ee1-9ccc-e3f39ff8eff2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:48.997592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34dd20-2053-4ee1-9ccc-e3f39ff8eff2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:48.997592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34dd20-2053-4ee1-9ccc-e3f39ff8eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.016398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5e34dd20-2053-4ee1-9ccc-e3f39ff8eff2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:49.017877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5e34dd20-2053-4ee1-9ccc-e3f39ff8eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:49.034960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54dbd1-28ca-42ac-89f9-8b6148e1aa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.035625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54dbd1-28ca-42ac-89f9-8b6148e1aa59 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.035906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54dbd1-28ca-42ac-89f9-8b6148e1aa59 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.036172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54dbd1-28ca-42ac-89f9-8b6148e1aa59 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.036935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54dbd1-28ca-42ac-89f9-8b6148e1aa59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:49.037242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54dbd1-28ca-42ac-89f9-8b6148e1aa59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:49.038719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54dbd1-28ca-42ac-89f9-8b6148e1aa59 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:49.049361 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 54/109] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:49.051029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5deafb-5d4a-404e-bbfc-65f31195a5e1 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.051330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5deafb-5d4a-404e-bbfc-65f31195a5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.051626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5deafb-5d4a-404e-bbfc-65f31195a5e1 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.053631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 55/110] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:49.054982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5deafb-5d4a-404e-bbfc-65f31195a5e1 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.104349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 56/111] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:49.106075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da109d0c-4adf-45cc-9944-982be8c970b6 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.106554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da109d0c-4adf-45cc-9944-982be8c970b6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.107013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da109d0c-4adf-45cc-9944-982be8c970b6 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.108603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 57/112] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:49.109486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da109d0c-4adf-45cc-9944-982be8c970b6 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.112154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:49.131865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:49.200070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5deafb-5d4a-404e-bbfc-65f31195a5e1 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:49.246174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da109d0c-4adf-45cc-9944-982be8c970b6 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:49.247739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736238d-6fd1-44b4-87b8-58b8ec6c7bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.248648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736238d-6fd1-44b4-87b8-58b8ec6c7bac None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.248993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736238d-6fd1-44b4-87b8-58b8ec6c7bac None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.249277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736238d-6fd1-44b4-87b8-58b8ec6c7bac None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.250952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736238d-6fd1-44b4-87b8-58b8ec6c7bac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:49.251265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736238d-6fd1-44b4-87b8-58b8ec6c7bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:49.252743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736238d-6fd1-44b4-87b8-58b8ec6c7bac None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:49.265939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 56/113] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:49.271158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321cfe-a757-4024-85d0-6da377002f64 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.271158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321cfe-a757-4024-85d0-6da377002f64 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.271158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321cfe-a757-4024-85d0-6da377002f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.290322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfc382-114d-4593-b6ad-7f0edbfb74fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.291500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc382-114d-4593-b6ad-7f0edbfb74fc None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.291636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d7321cfe-a757-4024-85d0-6da377002f64 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:49.292154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc382-114d-4593-b6ad-7f0edbfb74fc None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.292646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfc382-114d-4593-b6ad-7f0edbfb74fc None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.292827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d7321cfe-a757-4024-85d0-6da377002f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:49.294230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfc382-114d-4593-b6ad-7f0edbfb74fc None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:49.296013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfc382-114d-4593-b6ad-7f0edbfb74fc None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:49.297881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 58/114] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:49.299271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3b55d-cfc2-414c-9e74-6a40327ff37c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.303852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3b55d-cfc2-414c-9e74-6a40327ff37c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.304399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3b55d-cfc2-414c-9e74-6a40327ff37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.323873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-98c3b55d-cfc2-414c-9e74-6a40327ff37c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:49.325440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-98c3b55d-cfc2-414c-9e74-6a40327ff37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:49.373799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 57/115] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:49.379736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7321cfe-a757-4024-85d0-6da377002f64 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.403418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 59/116] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:49.404318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3b55d-cfc2-414c-9e74-6a40327ff37c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.409569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:49.417041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:49.480720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7321cfe-a757-4024-85d0-6da377002f64 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:49.508864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3b55d-cfc2-414c-9e74-6a40327ff37c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:49.523646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.524230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.524508 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.524781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] PATH_INFO: `/v3/users/552a02a71fda494eb71cf8a20e6e8090` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.556171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.557437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.558257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] RBAC: Authorizing `identity:get_user(user_id=552a02a71fda494eb71cf8a20e6e8090)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:49.559788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:49.561516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 58/117] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/users/552a02a71fda494eb71cf8a20e6e8090 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:49.562290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.562789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.563127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.564655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:49.566258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:49.569475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 60/118] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:49.570278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.710896 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:49.718624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:49.726485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42097fcf-2d6b-40c2-a518-2985afd23186 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:49.729507 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307f9b-7370-4f08-a7cd-3260339b710e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:49.762139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.763011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.763456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.763655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.764100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.764281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.764562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.764834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.777100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:49.782247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:49.783255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:49.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 59/119] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:49.789026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.841653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:49.843149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:49.909783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60bf44-a310-416e-bafe-1e61a64c8e7e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:49.933351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:49.947796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:49.948512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:49.948917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:49.949199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25/users/f27b32aa767849b1b542bbb8f3eeeb4e/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:49.951562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] RBAC: Authorizing `identity:create_grant(project_id=5331948968af49e2bac622c653dc3e25, user_id=f27b32aa767849b1b542bbb8f3eeeb4e, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:49.953006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:49.959053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 61/120] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:49.960238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.970802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 60/121] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:24:49 2022] PUT /identity/v3/projects/5331948968af49e2bac622c653dc3e25/users/f27b32aa767849b1b542bbb8f3eeeb4e/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:24:49.972394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:49.989098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.091904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b988f-9b73-4098-bd7b-3c2646f6ab2c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.124352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad087b-2562-47da-a72c-dd67e6997240 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.133145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.133807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.134087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.134395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.145949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:50.150835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:50.151721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:50.152467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 62/122] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:50.156568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.164074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.164732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.165018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.165304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.178966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:50.186172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:50.187110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:50.187921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:50.189052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 61/123] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:50.189325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.211076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.283214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498437c-28c9-4b26-991c-7faa32829cd5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.287248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.324234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.325179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.325535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.325878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.326619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:50.326993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.328042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765bc62-f046-4e88-b585-0d07116cb35a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.328577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.339821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 63/124] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.340456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.370204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.370772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.371056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.371338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.372594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.374177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.376111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 62/125] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.377044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.444971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.448704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dacf7-255d-45c6-8d74-374f82691279 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.485674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.486307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.486649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.486864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.487543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:50.487880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.489292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.489434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529fe62-53bf-494d-9266-9f57532cf6bc None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.498387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 64/126] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.499225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.526733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.527698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.528117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.528623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.529432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:50.529897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.531668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.537318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.542599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 63/127] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.543675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.609175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b12fa-2b22-44f7-b307-580b75cf367b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.644927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.645583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.646069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.646181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.647261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.648779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.651111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 65/128] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.651958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.655920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e420a-497f-499a-86de-42e5611f1198 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.683986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.693816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.695270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.695717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.696134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.697150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] RBAC: Adding query filter params (role.id=9d4a7bf6cf8243f1abad392422ff53f5, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=552a02a71fda494eb71cf8a20e6e8090) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:50.697791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.699497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.705955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 64/129] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/role_assignments?user.id=552a02a71fda494eb71cf8a20e6e8090&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=9d4a7bf6cf8243f1abad392422ff53f5 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.707071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.764394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeba27-d377-479e-b722-6657b1b7b93d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.782825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.802205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.802881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.803220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.803715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] PATH_INFO: `/v3/users/c35278cad6234455851a9ea018219e84` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.804832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] RBAC: Authorizing `identity:get_user(user_id=c35278cad6234455851a9ea018219e84)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.806457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.809483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 66/130] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/users/c35278cad6234455851a9ea018219e84 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.810253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.828855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d638c-e385-4c2b-8528-cee6ad4aba6a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.866692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.867872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.868467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.869229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.870771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.872648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.874770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.875921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:50.928671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:50.934638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce67337-335e-414d-a93f-5f1a59285b8e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:50.974992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:50.979663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:50.979663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:50.979663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:50.983605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:50.983605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:50.984776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 67/132] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:50.985591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:51.022636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99856ff8-cd21-40dc-b00d-ad6012743ac7 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:51.062149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:51.069563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:51.070231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:51.070516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:51.070789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:51.073119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:51.074632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:51.076398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 66/133] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:51.077764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:51.201305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7479d7-e59d-427f-b205-d48224c16da9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:51.252799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:51.253520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:51.253805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:51.254120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] PATH_INFO: `/v3/projects/f7e9106180e64d35883fab40097dbb31` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:51.255461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e9106180e64d35883fab40097dbb31)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:51.257018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:51.257815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db19fb5-f0a0-46ae-a525-669500ad5410 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:51.258728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 68/134] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:50 2022] GET /identity/v3/projects/f7e9106180e64d35883fab40097dbb31 => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:51.260671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:51.326481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:51.327167 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:51.327451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:51.327756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25/users/f27b32aa767849b1b542bbb8f3eeeb4e/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:51.332159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:51.336072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] RBAC: Authorizing `identity:create_grant(project_id=5331948968af49e2bac622c653dc3e25, user_id=f27b32aa767849b1b542bbb8f3eeeb4e, role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:51.340454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:51.371001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 67/135] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:24:51 2022] PUT /identity/v3/projects/5331948968af49e2bac622c653dc3e25/users/f27b32aa767849b1b542bbb8f3eeeb4e/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:24:51.371892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:51.471280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef78e3-20c6-4ab8-9e9e-3a409dc54453 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:51.495042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:51.554442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5e639-6090-4dc1-96e2-32d2473fa3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:51.555493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e639-6090-4dc1-96e2-32d2473fa3e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:51.556009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e639-6090-4dc1-96e2-32d2473fa3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:51.558020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e639-6090-4dc1-96e2-32d2473fa3e2 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25/users/c35278cad6234455851a9ea018219e84/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:51.562888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5e639-6090-4dc1-96e2-32d2473fa3e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5331948968af49e2bac622c653dc3e25, user_id=c35278cad6234455851a9ea018219e84, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:51.564685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5e639-6090-4dc1-96e2-32d2473fa3e2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:51.586456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 69/136] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:24:51 2022] PUT /identity/v3/projects/5331948968af49e2bac622c653dc3e25/users/c35278cad6234455851a9ea018219e84/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:24:51.588731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1f622-a2a7-4cdf-b5e6-80c99392cec2 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:51.589279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1f622-a2a7-4cdf-b5e6-80c99392cec2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:51.589279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1f622-a2a7-4cdf-b5e6-80c99392cec2 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:51.590832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 70/137] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:51.593473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8289e68-939f-4a6c-9270-58f7bb958f7a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:51.596609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5df05-7622-43ea-927c-3c2121a66b98 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:51.596895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5df05-7622-43ea-927c-3c2121a66b98 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:51.598448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5df05-7622-43ea-927c-3c2121a66b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:51.622481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-78a5df05-7622-43ea-927c-3c2121a66b98 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:51.624740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-78a5df05-7622-43ea-927c-3c2121a66b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:51.646531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:51.647786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:51.648540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:51.649018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] PATH_INFO: `/v3/projects/f7e9106180e64d35883fab40097dbb31/users/f27b32aa767849b1b542bbb8f3eeeb4e/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:51.652225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e9106180e64d35883fab40097dbb31, user_id=f27b32aa767849b1b542bbb8f3eeeb4e, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:51.654965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:51.677491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 68/138] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:24:51 2022] PUT /identity/v3/projects/f7e9106180e64d35883fab40097dbb31/users/f27b32aa767849b1b542bbb8f3eeeb4e/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:24:51.718294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 71/139] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:52.109878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:52.230868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af58b9-e487-414f-8c71-6f8ca2799629 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:52.252593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:52.271388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08b6a3-9568-4d5a-a69b-bf7384291d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:52.272103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08b6a3-9568-4d5a-a69b-bf7384291d64 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:52.272616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08b6a3-9568-4d5a-a69b-bf7384291d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:52.272902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08b6a3-9568-4d5a-a69b-bf7384291d64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:52.274090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08b6a3-9568-4d5a-a69b-bf7384291d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:52.277432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08b6a3-9568-4d5a-a69b-bf7384291d64 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:52.279432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 69/140] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:52.284825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5df05-7622-43ea-927c-3c2121a66b98 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:52.301484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f631df-c2a2-4354-a19e-1ca7fc93eb3a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:52.302081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f631df-c2a2-4354-a19e-1ca7fc93eb3a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:52.302548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f631df-c2a2-4354-a19e-1ca7fc93eb3a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:52.304126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 70/141] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:52.309345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b655fbb-6d83-4f8b-bc39-9f92bb1465f6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:52.309870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b655fbb-6d83-4f8b-bc39-9f92bb1465f6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:52.310339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b655fbb-6d83-4f8b-bc39-9f92bb1465f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:52.328724 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:52.329058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8b655fbb-6d83-4f8b-bc39-9f92bb1465f6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:52.331085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8b655fbb-6d83-4f8b-bc39-9f92bb1465f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:52.421920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5df05-7622-43ea-927c-3c2121a66b98 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:52.458531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:52.459373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:52.459789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:52.460187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:52.460954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:52.461836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 71/142] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:52.462742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:52.502280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 72/143] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:24:52 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:52.902269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b655fbb-6d83-4f8b-bc39-9f92bb1465f6 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:53.032015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b655fbb-6d83-4f8b-bc39-9f92bb1465f6 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:53.064695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:53.080225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:53.080883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:53.081172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:53.081453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:53.097028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:53.102243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:53.103277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:53.104279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 72/144] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:53.109426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:53.212795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:53.239013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9321254-004a-476d-801b-8ad66f5118ff None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:53.285232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:53.286664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:53.287159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:53.287709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:53.288663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:53.289175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:53.290852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:53.300573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 73/145] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:53.305794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:53.435625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dac5f-0152-4fc7-8ccc-49c738e3d1bc None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:53.476364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d0faa-026e-4beb-aa3d-52d83a73c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:53.476941 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d0faa-026e-4beb-aa3d-52d83a73c474 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:53.477219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d0faa-026e-4beb-aa3d-52d83a73c474 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:53.477490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d0faa-026e-4beb-aa3d-52d83a73c474 None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:53.478782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d0faa-026e-4beb-aa3d-52d83a73c474 None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:53.480399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d0faa-026e-4beb-aa3d-52d83a73c474 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:53.482022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 73/146] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:53 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:53.488136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:53.503027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591b201-395b-455f-821c-313e3d7de524 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:53.503641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591b201-395b-455f-821c-313e3d7de524 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:53.504098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591b201-395b-455f-821c-313e3d7de524 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:53.505888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 74/147] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:53.511961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879425-1505-4364-8ce4-152bd63790f0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:53.512516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879425-1505-4364-8ce4-152bd63790f0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:53.512967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879425-1505-4364-8ce4-152bd63790f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:53.543169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-09879425-1505-4364-8ce4-152bd63790f0 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:53.544408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-09879425-1505-4364-8ce4-152bd63790f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:53.645654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:53.659083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371fef-7f8c-44d9-a287-5212426319ab None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:53.695884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 75/148] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:53.725234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:53.725872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:53.726157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:53.726489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:53.741512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:53.822683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:53.824170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:53.949580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 74/149] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 05:24:53.955066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879425-1505-4364-8ce4-152bd63790f0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:54.074988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879425-1505-4364-8ce4-152bd63790f0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:54.118313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6123479-f4f9-4278-9ea0-4dcfa2432825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:54.119029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123479-f4f9-4278-9ea0-4dcfa2432825 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.119309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123479-f4f9-4278-9ea0-4dcfa2432825 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.119628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123479-f4f9-4278-9ea0-4dcfa2432825 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.120858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6123479-f4f9-4278-9ea0-4dcfa2432825 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:54.122356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6123479-f4f9-4278-9ea0-4dcfa2432825 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:54.124397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 76/150] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:53 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:54.131433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:54.164548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22c8cb-8d67-4171-92fd-1ca2cbee3507 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.164871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22c8cb-8d67-4171-92fd-1ca2cbee3507 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.165098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22c8cb-8d67-4171-92fd-1ca2cbee3507 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.166734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 77/151] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:54.172364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00b33-07c4-4906-8caf-4b71b572e82c None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.172650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00b33-07c4-4906-8caf-4b71b572e82c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.172922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00b33-07c4-4906-8caf-4b71b572e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.205544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8fa00b33-07c4-4906-8caf-4b71b572e82c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:54.206765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8fa00b33-07c4-4906-8caf-4b71b572e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:54.224576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:54.298365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:54.346311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a5a54-068c-4778-9b49-033a86291e0c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:54.355546 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 78/152] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:24:54.356440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00b33-07c4-4906-8caf-4b71b572e82c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:54.409902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48508167-ff10-4afc-a779-59727651d67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:54.410531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508167-ff10-4afc-a779-59727651d67f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.410810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508167-ff10-4afc-a779-59727651d67f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.411082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508167-ff10-4afc-a779-59727651d67f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.411857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48508167-ff10-4afc-a779-59727651d67f None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=f27b32aa767849b1b542bbb8f3eeeb4e) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:54.412166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48508167-ff10-4afc-a779-59727651d67f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:54.413613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48508167-ff10-4afc-a779-59727651d67f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:54.430583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 75/153] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:54 2022] GET /identity/v3/role_assignments?user.id=f27b32aa767849b1b542bbb8f3eeeb4e&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=69140820db3a4dfea9428bdde1203a39 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:24:54.482432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c86b5-4ff3-4a06-ba3f-e1343a89cbf3 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.482755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c86b5-4ff3-4a06-ba3f-e1343a89cbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.482981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c86b5-4ff3-4a06-ba3f-e1343a89cbf3 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.488859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 76/154] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:24:54.495727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8decc2e-c44e-47b8-85d5-d879aa830ab6 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.496015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8decc2e-c44e-47b8-85d5-d879aa830ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.496290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8decc2e-c44e-47b8-85d5-d879aa830ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.508946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00b33-07c4-4906-8caf-4b71b572e82c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:54.536062 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-d8decc2e-c44e-47b8-85d5-d879aa830ab6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:54.537301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d8decc2e-c44e-47b8-85d5-d879aa830ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:54.569207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:54.570008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.570647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.570913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.595479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:54.601644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:54.603827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:54.605737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 79/155] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:54.611596 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:54.658298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 77/156] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:54.683041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:54.782981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221cd7b-fbed-43a8-9932-4290d7adc24b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:54.823204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:54.823730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:54.824182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:54.824462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:54.825211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:54.825523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:54.826965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:54.836803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 80/157] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:54.844728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8decc2e-c44e-47b8-85d5-d879aa830ab6 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:54.872765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:54.919672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:55.046049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8decc2e-c44e-47b8-85d5-d879aa830ab6 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.095810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:55.106906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd3150-73a1-4ad2-8af9-5ec681f74719 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.118590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.119712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.120212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.120671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] PATH_INFO: `/v3/users/552a02a71fda494eb71cf8a20e6e8090` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.123704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] RBAC: Authorizing `identity:get_user(user_id=552a02a71fda494eb71cf8a20e6e8090)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:55.125376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:55.127193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 78/158] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:54 2022] GET /identity/v3/users/552a02a71fda494eb71cf8a20e6e8090 => generated 289 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:55.133756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:55.174248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdeb189-a737-4640-afcf-fa601cdfe81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.174248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeb189-a737-4640-afcf-fa601cdfe81a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.174248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeb189-a737-4640-afcf-fa601cdfe81a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.174248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeb189-a737-4640-afcf-fa601cdfe81a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.184514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdeb189-a737-4640-afcf-fa601cdfe81a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-6fdeb189-a737-4640-afcf-fa601cdfe81a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:55.190120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:55.191057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:55.192212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 81/159] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:24:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:24:55.192893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe86f7-dc9a-4e01-be9f-746ee4be8254 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.193191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe86f7-dc9a-4e01-be9f-746ee4be8254 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.193342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe86f7-dc9a-4e01-be9f-746ee4be8254 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.194723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 82/160] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:55.203243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe86f7-dc9a-4e01-be9f-746ee4be8254 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:55.271934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e9a9-747a-48f9-8d8e-20010b260126 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.275337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:55.326080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe86f7-dc9a-4e01-be9f-746ee4be8254 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.328370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75b6b0-a255-4089-9c69-2abe4cefe46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.329234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75b6b0-a255-4089-9c69-2abe4cefe46b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.329664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75b6b0-a255-4089-9c69-2abe4cefe46b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.330078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75b6b0-a255-4089-9c69-2abe4cefe46b None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.331435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75b6b0-a255-4089-9c69-2abe4cefe46b None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:55.333248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75b6b0-a255-4089-9c69-2abe4cefe46b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:55.335034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 79/161] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:55 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:55.337422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d53b4-783e-46d8-ba0b-2658abf53b0b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.337860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d53b4-783e-46d8-ba0b-2658abf53b0b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.338308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d53b4-783e-46d8-ba0b-2658abf53b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.365226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.366578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.367109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.367614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.368631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:55.369138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:55.370869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:55.377244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e61d53b4-783e-46d8-ba0b-2658abf53b0b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:55.378258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e61d53b4-783e-46d8-ba0b-2658abf53b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:55.381736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 83/162] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:24:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:55.382975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:55.408550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:55.497435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03272b06-9e24-46ea-b91d-455233e34bab None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.543138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.544348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.544828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.545289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.546048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 80/163] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:55.546893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d53b4-783e-46d8-ba0b-2658abf53b0b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:55.568281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:55.574094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:55.574994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:55.575805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:24:55.578207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 84/164] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:55.579283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:55.693339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:55.719138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a8f82-8596-4692-ae4e-977972b1f4ca None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.756355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.757471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.757999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.758472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.759972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:55.761955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:55.763813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 85/165] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:55 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:55.764998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:55.825516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d53b4-783e-46d8-ba0b-2658abf53b0b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.827184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:55.891592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823b3b-70dc-4f3a-840b-bd47b34645c2 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:55.933973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.934982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.935455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.935956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.937001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:55.937514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:55.939191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:55.941320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:55.942267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:55.942267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:55.942559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25/users/552a02a71fda494eb71cf8a20e6e8090/roles/9d4a7bf6cf8243f1abad392422ff53f5` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:55.949070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] RBAC: Authorizing `identity:create_grant(project_id=5331948968af49e2bac622c653dc3e25, user_id=552a02a71fda494eb71cf8a20e6e8090, role_id=9d4a7bf6cf8243f1abad392422ff53f5)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:55.949716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 86/166] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:55.950482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:55.950973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:55.963456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:55.988299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 81/167] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:24:55 2022] PUT /identity/v3/projects/5331948968af49e2bac622c653dc3e25/users/552a02a71fda494eb71cf8a20e6e8090/roles/9d4a7bf6cf8243f1abad392422ff53f5 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:24:55.989217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:56.069613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ab7-48e7-4425-b12c-f2f4fa5327ce None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:56.115870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5004568-3899-4868-bf34-f2c98a289490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:56.116910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004568-3899-4868-bf34-f2c98a289490 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:56.117405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004568-3899-4868-bf34-f2c98a289490 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:56.117938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004568-3899-4868-bf34-f2c98a289490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:56.204173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5004568-3899-4868-bf34-f2c98a289490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:56.206131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5004568-3899-4868-bf34-f2c98a289490 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:56.217898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b989980-cff4-4ead-94da-d610944cbf3a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:56.219172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:56.296943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:56.297618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:56.297902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:56.298179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] PATH_INFO: `/v3/users/c35278cad6234455851a9ea018219e84` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:56.304398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] RBAC: Authorizing `identity:get_user(user_id=c35278cad6234455851a9ea018219e84)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:56.305935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:56.308283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 82/168] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:55 2022] GET /identity/v3/users/c35278cad6234455851a9ea018219e84 => generated 247 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:56.309210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:56.329963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 87/169] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:56.336155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85cf9a-b6c9-4ac4-be66-e57a9b6938cb None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:56.336699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85cf9a-b6c9-4ac4-be66-e57a9b6938cb None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:56.337256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85cf9a-b6c9-4ac4-be66-e57a9b6938cb None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:56.339023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 88/170] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:56.340130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85cf9a-b6c9-4ac4-be66-e57a9b6938cb None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:56.371189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:56.463337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835672f8-fb89-4e56-b8f3-36a019074875 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:56.476355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85cf9a-b6c9-4ac4-be66-e57a9b6938cb None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:56.492451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:56.530917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034d643-171c-42c5-b5cf-d6da73446348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:56.531568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d643-171c-42c5-b5cf-d6da73446348 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:56.531865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d643-171c-42c5-b5cf-d6da73446348 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:56.532134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d643-171c-42c5-b5cf-d6da73446348 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:56.541332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:56.542714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:56.542996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:56.543265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:56.559257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034d643-171c-42c5-b5cf-d6da73446348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d034d643-171c-42c5-b5cf-d6da73446348 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:56.565708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:56.566617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:56.567382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 83/171] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:56.568655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb668-c3d0-4040-ae1f-27f2d92a3c4d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:56.568655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb668-c3d0-4040-ae1f-27f2d92a3c4d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:56.568930 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb668-c3d0-4040-ae1f-27f2d92a3c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:56.618048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ca7bb668-c3d0-4040-ae1f-27f2d92a3c4d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:56.619299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ca7bb668-c3d0-4040-ae1f-27f2d92a3c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:56.640497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:56.642679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:56.760522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 84/172] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:56.761625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bb668-c3d0-4040-ae1f-27f2d92a3c4d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:56.803500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:56.836144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 89/173] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:56.837007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:56.854650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:56.953118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bb668-c3d0-4040-ae1f-27f2d92a3c4d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.024427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f808-a8e0-48d2-b5b2-2810c46dbc13 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.025498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.026342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.026752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.027200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.030027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:57.031677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:57.039051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 85/174] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:56 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:57.040118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:57.106515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.107391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.107888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.108290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.109099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:57.109532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:57.111056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:57.130534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 90/175] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:57.131644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:57.192850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:57.211992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b1dad-b15d-40c8-9804-fc186c45caf6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.234680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:57.249768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.250653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.250963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.251616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.252656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:57.254424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:57.259070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 86/176] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:57 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:57.259070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:57.279615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf525c4d-2433-41f1-934f-dd4725533c91 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.318879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63f90a-bca6-4429-85e2-704fa4b30c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.319901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f90a-bca6-4429-85e2-704fa4b30c15 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.320367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f90a-bca6-4429-85e2-704fa4b30c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.320835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f90a-bca6-4429-85e2-704fa4b30c15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.321846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63f90a-bca6-4429-85e2-704fa4b30c15 None admin] RBAC: Adding query filter params (role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=f27b32aa767849b1b542bbb8f3eeeb4e) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:57.322357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63f90a-bca6-4429-85e2-704fa4b30c15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:57.324040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63f90a-bca6-4429-85e2-704fa4b30c15 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:57.331147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 91/177] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:57 2022] GET /identity/v3/role_assignments?user.id=f27b32aa767849b1b542bbb8f3eeeb4e&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:57.332817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d991e-afaa-421a-bd20-1310b62a3145 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.333222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d991e-afaa-421a-bd20-1310b62a3145 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.333561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d991e-afaa-421a-bd20-1310b62a3145 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.334837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 92/178] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:57.335631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d991e-afaa-421a-bd20-1310b62a3145 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:57.392589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f8e2d-ef7c-430b-adc5-510a89eb4130 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.431421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.432239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.432576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.432910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] PATH_INFO: `/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.434376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] RBAC: Authorizing `identity:get_user(user_id=f27b32aa767849b1b542bbb8f3eeeb4e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:57.436081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:57.442465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 87/179] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:57 2022] GET /identity/v3/users/f27b32aa767849b1b542bbb8f3eeeb4e => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:57.442465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:57.467894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:57.477856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d991e-afaa-421a-bd20-1310b62a3145 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.530917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2f67-11e5-4839-b66d-4a8d69817e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.531438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2f67-11e5-4839-b66d-4a8d69817e57 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.531739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2f67-11e5-4839-b66d-4a8d69817e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.532005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2f67-11e5-4839-b66d-4a8d69817e57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.532739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e2f67-11e5-4839-b66d-4a8d69817e57 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=c35278cad6234455851a9ea018219e84) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:57.533042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e2f67-11e5-4839-b66d-4a8d69817e57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:57.534545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e2f67-11e5-4839-b66d-4a8d69817e57 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:57.536633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:57.544068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 93/180] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:57 2022] GET /identity/v3/role_assignments?user.id=c35278cad6234455851a9ea018219e84&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:57.545629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b543d7-f2d5-41b3-a6ac-d817840d95cb None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.545983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b543d7-f2d5-41b3-a6ac-d817840d95cb None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.546320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b543d7-f2d5-41b3-a6ac-d817840d95cb None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.547737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 94/181] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:57.548768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e462a-9f9d-4d3f-8a35-6044fa409e24 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.549134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e462a-9f9d-4d3f-8a35-6044fa409e24 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.549495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e462a-9f9d-4d3f-8a35-6044fa409e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.576695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-072e462a-9f9d-4d3f-8a35-6044fa409e24 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:57.579263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-072e462a-9f9d-4d3f-8a35-6044fa409e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:57.585305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92d69d-35f3-4bcb-9138-8c70904b4408 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.639479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.640495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.641349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.642006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.657622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:57.665922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:57.666891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:24:57.668323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:57.668933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 88/182] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:57.669822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:57.675215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 95/183] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:57.675770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b98ca9-31ce-44b5-a044-327dbbb5f09e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.676156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b98ca9-31ce-44b5-a044-327dbbb5f09e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.676441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b98ca9-31ce-44b5-a044-327dbbb5f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.696538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b1b98ca9-31ce-44b5-a044-327dbbb5f09e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:57.697742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b1b98ca9-31ce-44b5-a044-327dbbb5f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:57.782989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:57.783311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 96/184] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:57.786398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b98ca9-31ce-44b5-a044-327dbbb5f09e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:57.804061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327fb99-8847-418e-a2a5-76855cac2076 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.855023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:57.855493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:57.855885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:57.856229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:57.862704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:57.945709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b98ca9-31ce-44b5-a044-327dbbb5f09e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:57.967604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:57.970576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.014312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.014955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.015257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.015652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.016443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:58.016749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.018187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.030450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 97/185] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:58.031417 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.082124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:58.108154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 89/186] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:58.109455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.163851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cb9cc-66ae-48db-88bd-2a0c98bcc678 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.186071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:58.203774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.204427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.204708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.204982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.206216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.207813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.209519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 98/187] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:58.210280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.226559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c60e0-ae8a-4fed-a0eb-c29347ab8cb1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.262148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.262803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.263079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.263347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] PATH_INFO: `/v3/users/1cd348b6a2d4444695294f0bb486d625` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.279046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] RBAC: Authorizing `identity:get_user(user_id=1cd348b6a2d4444695294f0bb486d625)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.280533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.282179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 90/188] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/users/1cd348b6a2d4444695294f0bb486d625 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:58.282942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.320554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ffa98-42d0-4d6e-a1be-c346eef0a2f6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.328080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:58.357520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.358382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.358801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.359209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] PATH_INFO: `/v3/projects/f7e9106180e64d35883fab40097dbb31` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.360545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] RBAC: Authorizing `identity:get_project(project_id=f7e9106180e64d35883fab40097dbb31)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.362166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.363992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 99/189] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/projects/f7e9106180e64d35883fab40097dbb31 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:58.364889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.392354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01618e21-eea7-495b-8f51-07811b5c85a9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.420898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:58.427496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.428175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.428456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.428728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.439400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:58.444537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:58.445389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:58.446181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 91/190] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:24:58.447523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.470866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362ed2-84bb-449d-9aec-eac36020fd3a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.506156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.506784 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.507030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.507301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.507984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.509479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.537661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 100/191] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:24:58 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:24:58.538405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.568879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133c4f-3517-4491-94b3-9d80986654b2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.570616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:58.608886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d7fe0-a0c8-4a92-9e2f-b978b61050db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.609476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d7fe0-a0c8-4a92-9e2f-b978b61050db None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.609760 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d7fe0-a0c8-4a92-9e2f-b978b61050db None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.610036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d7fe0-a0c8-4a92-9e2f-b978b61050db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.669341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ad1de-8d50-4029-9615-2c4a5c4ce49c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.684493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:58.689193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d7fe0-a0c8-4a92-9e2f-b978b61050db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.690700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d7fe0-a0c8-4a92-9e2f-b978b61050db None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.715409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.716300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.716583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.716852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.717697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=f7e9106180e64d35883fab40097dbb31, user.id=f27b32aa767849b1b542bbb8f3eeeb4e) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:58.717898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.719379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.726178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 101/192] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/role_assignments?user.id=f27b32aa767849b1b542bbb8f3eeeb4e&scope.project.id=f7e9106180e64d35883fab40097dbb31&role.id=69140820db3a4dfea9428bdde1203a39 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:58.726937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.824282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 92/193] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:58.826949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd3dd-9b4d-47e1-ab77-9ec349629235 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.827420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd3dd-9b4d-47e1-ab77-9ec349629235 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.828001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd3dd-9b4d-47e1-ab77-9ec349629235 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.829476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 93/194] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:24:58.832234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd3dd-9b4d-47e1-ab77-9ec349629235 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:58.836622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:58.858272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413b2ed-7773-4fed-817e-628ed539f517 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:58.913282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633a563-3ece-4180-81c5-b93708373391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:58.914281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633a563-3ece-4180-81c5-b93708373391 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.914571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633a563-3ece-4180-81c5-b93708373391 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.914833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633a563-3ece-4180-81c5-b93708373391 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.915604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633a563-3ece-4180-81c5-b93708373391 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:58.918715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633a563-3ece-4180-81c5-b93708373391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:58.920190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633a563-3ece-4180-81c5-b93708373391 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:58.933279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 102/195] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:58.935005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695e501-b70a-4e0c-abed-b2dddbdc3930 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:58.935355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695e501-b70a-4e0c-abed-b2dddbdc3930 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:58.935736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695e501-b70a-4e0c-abed-b2dddbdc3930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:58.959437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e695e501-b70a-4e0c-abed-b2dddbdc3930 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:24:58.961380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e695e501-b70a-4e0c-abed-b2dddbdc3930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:24:58.983019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd3dd-9b4d-47e1-ab77-9ec349629235 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:59.003430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:59.024572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:59.025435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:59.025713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:59.026167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:59.027099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:59.028623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:59.030585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 94/196] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:58 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:24:59.031403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:59.055250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 103/197] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:24:59.056092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695e501-b70a-4e0c-abed-b2dddbdc3930 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:59.155561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:59.196548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec86129-6ce2-4bb3-8fb3-e0373e02e73a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:59.230250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695e501-b70a-4e0c-abed-b2dddbdc3930 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:59.269141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:59.269769 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:59.270049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:59.270149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:59.270337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:59.270797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:59.271056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=1cd348b6a2d4444695294f0bb486d625) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:59.271165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:59.271315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] PATH_INFO: `/v3/users/1cd348b6a2d4444695294f0bb486d625` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:59.271402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:59.272850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:59.276304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] RBAC: Authorizing `identity:get_user(user_id=1cd348b6a2d4444695294f0bb486d625)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:59.277776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:59.279306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 104/198] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/role_assignments?user.id=1cd348b6a2d4444695294f0bb486d625&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:59.279487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 95/199] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/users/1cd348b6a2d4444695294f0bb486d625 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:59.285486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:59.334766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:59.341474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:59.432741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75322-a49e-483a-8a2e-c8383d6a00a2 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:59.510273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:59.510906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:59.511169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:59.511432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:59.544241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ef215-47e7-4619-9867-117bd90fcd47 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:59.594599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:59.596138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:59.608503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:59.609160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:59.609437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:59.609711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:59.625585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:59.627817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:59.634961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:24:59.642695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:24:59.643564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:24:59.644371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:24:59.651888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 96/200] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:24:59.652879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:59.706072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 105/201] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:24:59.712431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:59.842968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41def9-de95-4d5a-ab74-54cb5fc0e279 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:59.884182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:24:59.885683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:59.886828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:59.887324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:59.887836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:59.889919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:59.892258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:59.896570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4a1ac-4065-4dc3-bf0e-4c7827324f14 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:24:59.898696 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 106/202] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:59.904137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:24:59.954900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:24:59.955585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:24:59.955871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:24:59.956200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:24:59.957287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:24:59.957743 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:24:59.959211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:24:59.969089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 97/203] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:24:59.974892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:00.059071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:00.105081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624d4ec-9524-408f-8c4a-4c5d4ce0257e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:00.141133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:00.141847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:00.142117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:00.142397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] PATH_INFO: `/v3/users/552a02a71fda494eb71cf8a20e6e8090` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:00.143543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] RBAC: Authorizing `identity:get_user(user_id=552a02a71fda494eb71cf8a20e6e8090)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:00.145408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:00.147344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 98/204] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/users/552a02a71fda494eb71cf8a20e6e8090 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:00.153269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:00.197843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f679a92-f78e-4150-8ecc-22d6819d2b27 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:00.262395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4746c0-455f-4241-8c77-ef3986966f66 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:00.292130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:00.309236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:00.310174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:00.310466 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:00.310743 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:00.323345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:00.328859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:00.328859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:00.328859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:00.328859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] RBAC: Adding query filter params (role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=1cd348b6a2d4444695294f0bb486d625) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:00.328859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:00.328859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:00.340388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 107/205] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:24:59 2022] GET /identity/v3/role_assignments?user.id=1cd348b6a2d4444695294f0bb486d625&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 259 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:00.427939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:00.429443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:00.542017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 99/206] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:00.548136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:00.626073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:00.665352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1ae56-173b-492d-a731-3c8007a43e0c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:00.703723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97862422-8460-4e5a-a7c3-62e56a1eebcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:00.704373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862422-8460-4e5a-a7c3-62e56a1eebcd None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:00.704655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862422-8460-4e5a-a7c3-62e56a1eebcd None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:00.704928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862422-8460-4e5a-a7c3-62e56a1eebcd None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:00.706068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97862422-8460-4e5a-a7c3-62e56a1eebcd None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:00.707537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97862422-8460-4e5a-a7c3-62e56a1eebcd None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:00.709218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 108/207] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:00 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:00.715278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:00.756147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:00.858220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca03ed0-221c-40f1-8503-52bee55a2273 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:00.893169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc51df-f2d1-4fc6-95a7-b5e69c132405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:00.893801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc51df-f2d1-4fc6-95a7-b5e69c132405 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:00.894086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc51df-f2d1-4fc6-95a7-b5e69c132405 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:00.894360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc51df-f2d1-4fc6-95a7-b5e69c132405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:00.895101 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc51df-f2d1-4fc6-95a7-b5e69c132405 None admin] RBAC: Adding query filter params (role.id=9d4a7bf6cf8243f1abad392422ff53f5, scope.project.id=5331948968af49e2bac622c653dc3e25, user.id=552a02a71fda494eb71cf8a20e6e8090) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:00.895410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc51df-f2d1-4fc6-95a7-b5e69c132405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:00.896918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc51df-f2d1-4fc6-95a7-b5e69c132405 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:00.908416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 100/208] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:00 2022] GET /identity/v3/role_assignments?user.id=552a02a71fda494eb71cf8a20e6e8090&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=9d4a7bf6cf8243f1abad392422ff53f5 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:00.958293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd041f1-0825-46d8-814d-e7661e0e398a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:00.958740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd041f1-0825-46d8-814d-e7661e0e398a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:00.959115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd041f1-0825-46d8-814d-e7661e0e398a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:00.960950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 109/209] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:00.966924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0274e-6281-4c48-90ac-a5526dae194b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:00.967213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0274e-6281-4c48-90ac-a5526dae194b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:00.967490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0274e-6281-4c48-90ac-a5526dae194b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:00.992655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-d5b0274e-6281-4c48-90ac-a5526dae194b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:00.993934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d5b0274e-6281-4c48-90ac-a5526dae194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:01.094733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 101/210] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:01.216508 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26d2ad-73f1-4e9c-94f7-e451d254d10c None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.216861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26d2ad-73f1-4e9c-94f7-e451d254d10c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.217067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26d2ad-73f1-4e9c-94f7-e451d254d10c None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.218729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 110/211] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:01.224386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fb814-517e-4419-9006-8c63f3cd2ad8 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.224679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fb814-517e-4419-9006-8c63f3cd2ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.224954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fb814-517e-4419-9006-8c63f3cd2ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.249381 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f42fb814-517e-4419-9006-8c63f3cd2ad8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:01.250812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f42fb814-517e-4419-9006-8c63f3cd2ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:01.270732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:01.324763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 102/212] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:01.385839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26d2ad-73f1-4e9c-94f7-e451d254d10c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:01.500343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26d2ad-73f1-4e9c-94f7-e451d254d10c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:01.533630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:01.539275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:01.540469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.541106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.541673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.553792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:01.559387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:01.560324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:01.561106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:01.564020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 111/213] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:01.572380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fb814-517e-4419-9006-8c63f3cd2ad8 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:01.616263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:01.683259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fb814-517e-4419-9006-8c63f3cd2ad8 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:01.718119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f8dff-ec8f-497d-88f1-ee2128ee42f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:01.718781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f8dff-ec8f-497d-88f1-ee2128ee42f4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.719271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f8dff-ec8f-497d-88f1-ee2128ee42f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.719541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f8dff-ec8f-497d-88f1-ee2128ee42f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.720388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f8dff-ec8f-497d-88f1-ee2128ee42f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:01.720691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f8dff-ec8f-497d-88f1-ee2128ee42f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:01.722397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f8dff-ec8f-497d-88f1-ee2128ee42f4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:01.730252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba562d2-ac95-4344-b164-400b4d108f82 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:01.735630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 103/214] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:01.735630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822b51-49bf-4efd-9a2f-660f1ce54110 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.735630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822b51-49bf-4efd-9a2f-660f1ce54110 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.735630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822b51-49bf-4efd-9a2f-660f1ce54110 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.735630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 104/215] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:01.736549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27822b51-49bf-4efd-9a2f-660f1ce54110 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:01.752471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:01.775671 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e84ca2-f87e-480b-93fe-231a47f1decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:01.776302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e84ca2-f87e-480b-93fe-231a47f1decc None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.776583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e84ca2-f87e-480b-93fe-231a47f1decc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.776854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e84ca2-f87e-480b-93fe-231a47f1decc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.777965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e84ca2-f87e-480b-93fe-231a47f1decc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:01.779445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e84ca2-f87e-480b-93fe-231a47f1decc None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:01.781086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 112/216] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:01.782234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c342f8-cf6d-426b-944e-1295cfb0856b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.782468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c342f8-cf6d-426b-944e-1295cfb0856b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.782744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c342f8-cf6d-426b-944e-1295cfb0856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.800333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-25c342f8-cf6d-426b-944e-1295cfb0856b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:01.801542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-25c342f8-cf6d-426b-944e-1295cfb0856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:01.847160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27822b51-49bf-4efd-9a2f-660f1ce54110 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:01.860039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:01.873509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 113/217] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:01.874388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c342f8-cf6d-426b-944e-1295cfb0856b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:01.892917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:01.897358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:01.897358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:01.897358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] PATH_INFO: `/v3/users/65c516402d8f4077a809716307fa9d2d` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:01.915053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] RBAC: Authorizing `identity:get_user(user_id=65c516402d8f4077a809716307fa9d2d)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:01.917108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:01.923622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 105/218] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:01 2022] GET /identity/v3/users/65c516402d8f4077a809716307fa9d2d => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:01.924893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.004134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:02.016164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c342f8-cf6d-426b-944e-1295cfb0856b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.035768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5d9b1-ec1c-47f6-832b-9b7498757736 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.060818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.061944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.062416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.063176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.064294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:02.067818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:02.070441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.071063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.071369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.071677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.104059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 114/219] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:01 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:02.142980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.156773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:02.158317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:02.196818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:02.237956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:02.260445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155d4d6-e166-46c5-b90a-da44fc5ce2be None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.276841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 106/220] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:02.279436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.298360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.299011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.299289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.299557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.313449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:02.318563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:02.319453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:02.320458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 115/221] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:02.326330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.411536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70bd5f-7581-43fa-bf44-2af62ab99e90 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.454527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55435b75-c89b-45e8-8d82-d0a4e9bf6412 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.458135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.458788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.459075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.459333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.460499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:02.461780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:02.462814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:02.465038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 107/222] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:02 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:02.472449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.499996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.501215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.501649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.502066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.502894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:02.503934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:02.505658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:02.518457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 116/223] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:02.524131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.525594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:02.609262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef73bf-4efb-4cd4-9e2b-945189c8a13c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.630113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb2c52-41af-417d-843d-21e0555b8632 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.650682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5066c6c-b238-48f6-93c9-4da51d4ef826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.651305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066c6c-b238-48f6-93c9-4da51d4ef826 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.651604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066c6c-b238-48f6-93c9-4da51d4ef826 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.651882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066c6c-b238-48f6-93c9-4da51d4ef826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.652620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5066c6c-b238-48f6-93c9-4da51d4ef826 None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=65c516402d8f4077a809716307fa9d2d) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:02.652915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5066c6c-b238-48f6-93c9-4da51d4ef826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:02.654448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5066c6c-b238-48f6-93c9-4da51d4ef826 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:02.660513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 108/224] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:02 2022] GET /identity/v3/role_assignments?user.id=65c516402d8f4077a809716307fa9d2d&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=69140820db3a4dfea9428bdde1203a39 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:02.666442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a773f-4393-40f7-b37c-d3ebf31081a7 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.666722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a773f-4393-40f7-b37c-d3ebf31081a7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.666994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a773f-4393-40f7-b37c-d3ebf31081a7 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.668616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 109/225] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:02.674280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cd942-dddd-4251-a53b-bc8a5b439755 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.674494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cd942-dddd-4251-a53b-bc8a5b439755 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.674773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cd942-dddd-4251-a53b-bc8a5b439755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.684421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.685102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.685387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.685726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] PATH_INFO: `/v3/users/1cd348b6a2d4444695294f0bb486d625` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.692536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=1cd348b6a2d4444695294f0bb486d625)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:02.694057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:02.694713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-295cd942-dddd-4251-a53b-bc8a5b439755 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:02.695981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-295cd942-dddd-4251-a53b-bc8a5b439755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:02.696237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 117/226] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:02 2022] GET /identity/v3/users/1cd348b6a2d4444695294f0bb486d625 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:02.704292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.755333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:02.792785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 110/227] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:02.870036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f5a15-9c78-4250-b426-9f961ccd4ad6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:02.923294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:02.924344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:02.924862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:02.925330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:02.926672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:02.928600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:02.930475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 118/228] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:02 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:02.936476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cd942-dddd-4251-a53b-bc8a5b439755 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:02.973581 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:03.049200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cd942-dddd-4251-a53b-bc8a5b439755 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:03.096336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:03.097454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:03.098359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:03.098826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1/users/1cd348b6a2d4444695294f0bb486d625/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:03.104975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] RBAC: Authorizing `identity:create_grant(project_id=f33b9d002631429689e361f259c154d1, user_id=1cd348b6a2d4444695294f0bb486d625, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:03.106710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:03.114049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:03.128730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 111/229] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:02 2022] PUT /identity/v3/projects/f33b9d002631429689e361f259c154d1/users/1cd348b6a2d4444695294f0bb486d625/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:25:03.238771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132521bc-d0cc-4e0b-a983-4d242006371f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:03.242311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:03.279568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:03.280911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:03.281420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:03.281882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:03.294012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:03.299116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:03.300071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:03.300869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:03.303245 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 119/230] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:03.312781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:03.426998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:03.435812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8210f1-118a-49db-a50b-926360c58ccc None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:03.476051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:03.476682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:03.476962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:03.477285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:03.478214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:03.478695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:03.480133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:03.488821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 112/231] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:03.494081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:03.626224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d1f0a-e328-4d2d-b905-4ba6f2db2c4e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:03.671485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:03.672933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:03.673387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:03.673832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] PATH_INFO: `/v3/users/1cd348b6a2d4444695294f0bb486d625` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:03.675183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] RBAC: Authorizing `identity:get_user(user_id=1cd348b6a2d4444695294f0bb486d625)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:03.677011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:03.679220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 120/232] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:03 2022] GET /identity/v3/users/1cd348b6a2d4444695294f0bb486d625 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:03.690620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:04.101641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785ab13-f6fc-49d1-89c4-618febec7ad8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:04.244389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d1ead-c2a6-47bc-bfb3-4131e2c6f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:04.245319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1ead-c2a6-47bc-bfb3-4131e2c6f8dd None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:04.245742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1ead-c2a6-47bc-bfb3-4131e2c6f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:04.246153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1ead-c2a6-47bc-bfb3-4131e2c6f8dd None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:04.247530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d1ead-c2a6-47bc-bfb3-4131e2c6f8dd None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:04.249199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d1ead-c2a6-47bc-bfb3-4131e2c6f8dd None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:04.251037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 113/233] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:03 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 561 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:04.266610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:04.343076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:04.501785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d1e97-2663-4029-93e0-c0e38a52a36b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:04.564680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7ce9e-568c-481d-a333-6ce051198bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:04.565727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7ce9e-568c-481d-a333-6ce051198bcc None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:04.566223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7ce9e-568c-481d-a333-6ce051198bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:04.566707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7ce9e-568c-481d-a333-6ce051198bcc None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1/users/1cd348b6a2d4444695294f0bb486d625/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:04.569371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7ce9e-568c-481d-a333-6ce051198bcc None admin] RBAC: Authorizing `identity:create_grant(project_id=f33b9d002631429689e361f259c154d1, user_id=1cd348b6a2d4444695294f0bb486d625, role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:04.571184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7ce9e-568c-481d-a333-6ce051198bcc None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:04.588956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 121/234] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:04 2022] PUT /identity/v3/projects/f33b9d002631429689e361f259c154d1/users/1cd348b6a2d4444695294f0bb486d625/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:04.949359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8e94-4ca0-4fa2-a967-4a1a6ca49cc9 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:04.949907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8e94-4ca0-4fa2-a967-4a1a6ca49cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:04.950337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8e94-4ca0-4fa2-a967-4a1a6ca49cc9 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:04.952136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 114/235] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:04.957145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2117638-badf-48e3-ac46-25b31bc1d68c None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:04.958003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2117638-badf-48e3-ac46-25b31bc1d68c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:04.958486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2117638-badf-48e3-ac46-25b31bc1d68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:04.979238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d2117638-badf-48e3-ac46-25b31bc1d68c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:04.980814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d2117638-badf-48e3-ac46-25b31bc1d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:05.052628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35492a1-58fc-4087-b59e-084a4d334852 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.052957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35492a1-58fc-4087-b59e-084a4d334852 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.053275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35492a1-58fc-4087-b59e-084a4d334852 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.053474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 122/236] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:05.059660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 115/237] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:05.068314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe2559-8171-4168-bef8-861baa8055c5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.068847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe2559-8171-4168-bef8-861baa8055c5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.069333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe2559-8171-4168-bef8-861baa8055c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.089440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6afe2559-8171-4168-bef8-861baa8055c5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:05.090992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6afe2559-8171-4168-bef8-861baa8055c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:05.110533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:05.144810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803db49-9d0f-43fb-bf18-0127e0ebeb4d None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.145350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803db49-9d0f-43fb-bf18-0127e0ebeb4d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.145350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803db49-9d0f-43fb-bf18-0127e0ebeb4d None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.146943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 116/238] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:05.152293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5cfa-ce5c-47da-b59c-32b947e4843b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.152541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5cfa-ce5c-47da-b59c-32b947e4843b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.152797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5cfa-ce5c-47da-b59c-32b947e4843b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.176220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 123/239] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:05.182302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-480b5cfa-ce5c-47da-b59c-32b947e4843b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:05.183745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-480b5cfa-ce5c-47da-b59c-32b947e4843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:05.228450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:05.305938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 117/240] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:05.418113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe2559-8171-4168-bef8-861baa8055c5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:05.469216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b5cfa-ce5c-47da-b59c-32b947e4843b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:05.559681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe2559-8171-4168-bef8-861baa8055c5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:05.617955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b5cfa-ce5c-47da-b59c-32b947e4843b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:05.622543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:05.623189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.623463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.623763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.635392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:05.640422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:05.641301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:05.642075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 124/241] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:05.642738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:05.658022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:05.658666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.658937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.659634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.665850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:05.673148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:05.678779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:05.679668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:05.680465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:05.681098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 118/242] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:05.683628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:05.699813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:05.761504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c01c4-51a4-418e-b0ac-f29a247a91df None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:05.786726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8749d-3318-436b-bb2c-62c308bab707 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:05.796777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:05.797421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.797698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.797966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.799077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:05.800584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:05.802309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 125/243] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:05.808042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:05.822415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c4ab9-083a-4327-a743-a691d6a09cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:05.823039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c4ab9-083a-4327-a743-a691d6a09cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.823311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c4ab9-083a-4327-a743-a691d6a09cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.823566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c4ab9-083a-4327-a743-a691d6a09cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.824277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c4ab9-083a-4327-a743-a691d6a09cb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:05.824558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c4ab9-083a-4327-a743-a691d6a09cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:05.825935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c4ab9-083a-4327-a743-a691d6a09cb1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:05.834851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 119/244] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:05.835992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41539d8-bdca-497e-8dc6-be1765a425c1 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.836276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41539d8-bdca-497e-8dc6-be1765a425c1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.836548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41539d8-bdca-497e-8dc6-be1765a425c1 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.842051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 120/245] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:05.842051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41539d8-bdca-497e-8dc6-be1765a425c1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:05.900523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:05.924414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c454b3-188b-4355-a38b-7f18441822e5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:05.925858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:05.955707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41539d8-bdca-497e-8dc6-be1765a425c1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:05.959951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:05.961128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.961581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.962366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.963799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:05.964276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:05.965992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:05.975561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 126/246] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:05.976689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:05.992270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29588473-f9d0-4095-928e-f25bf0dd4d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:05.992891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29588473-f9d0-4095-928e-f25bf0dd4d5f None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:05.993222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29588473-f9d0-4095-928e-f25bf0dd4d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:05.993960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29588473-f9d0-4095-928e-f25bf0dd4d5f None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:05.994324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29588473-f9d0-4095-928e-f25bf0dd4d5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:05.995787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29588473-f9d0-4095-928e-f25bf0dd4d5f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:06.023642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 121/247] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:05 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:06.023642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc9f3-f38a-4097-bbb7-708285ee77fc None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.023642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc9f3-f38a-4097-bbb7-708285ee77fc None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.023642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc9f3-f38a-4097-bbb7-708285ee77fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.049416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-009cc9f3-f38a-4097-bbb7-708285ee77fc None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:06.050660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-009cc9f3-f38a-4097-bbb7-708285ee77fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:06.126548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d425d-2325-4d37-8eac-621babe5e6e0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:06.127878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 122/248] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:06.128549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cc9f3-f38a-4097-bbb7-708285ee77fc None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:06.171562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:06.172224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.172500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.172769 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] PATH_INFO: `/v3/users/65c516402d8f4077a809716307fa9d2d` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.174683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] RBAC: Authorizing `identity:get_user(user_id=65c516402d8f4077a809716307fa9d2d)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:06.175353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:06.177121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 127/249] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:05 2022] GET /identity/v3/users/65c516402d8f4077a809716307fa9d2d => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:06.188582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:06.204103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:06.206384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:06.257613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cc9f3-f38a-4097-bbb7-708285ee77fc None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:06.295527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:06.296204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.296484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.296755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] PATH_INFO: `/v3/users/11838d29f61a40cc941851ad5a68719f` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.312364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] RBAC: Authorizing `identity:get_user(user_id=11838d29f61a40cc941851ad5a68719f)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:06.313754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:06.316538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 123/250] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:06 2022] GET /identity/v3/users/11838d29f61a40cc941851ad5a68719f => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:06.318950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adb74d-1c02-49da-ba4e-6b61d17209d6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:06.321527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:06.359417 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:06.360470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.360911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.361439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.362803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:06.364535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:06.366810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 128/251] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:06 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:06.373539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:06.427490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3e71d-382c-40ec-84fd-ca42219384af None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:06.450699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:06.472037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:06.473176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:06.474034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.474142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.474339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.502763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db876e5b-982e-40c1-99ea-f0fe6340cabd None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:06.550053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:06.551041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.551635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.552083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1/users/65c516402d8f4077a809716307fa9d2d/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.554341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] RBAC: Authorizing `identity:create_grant(project_id=f33b9d002631429689e361f259c154d1, user_id=65c516402d8f4077a809716307fa9d2d, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:06.556167 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:06.560105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:06.562650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:06.576806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 129/252] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:06 2022] PUT /identity/v3/projects/f33b9d002631429689e361f259c154d1/users/65c516402d8f4077a809716307fa9d2d/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:25:06.578861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:06.712235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 124/253] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:06.721165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:06.757821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:06.766471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a197d-73c4-490b-8df1-939702626c9a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:06.828339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:06.829351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.829810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.830256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.844456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:06.852679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:06.852916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb68f-56da-47eb-b482-732b3666b21f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:06.859196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:06.860115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:06.860909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:06.863646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 130/254] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:06.869923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:06.889226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:06.890109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:06.890109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:06.890391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:06.891504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:06.894658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:06.894658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 125/255] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:06 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:06.909965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:07.042998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7f2f4-6153-4025-95a2-a6f9fc75d1c6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:07.074547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:07.094422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba99919-3957-40b5-a522-1d6ed248e2ff None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:07.109731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182acb9-28e0-4edf-bc05-7cc2c0b3cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:07.110357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182acb9-28e0-4edf-bc05-7cc2c0b3cc18 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:07.110631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182acb9-28e0-4edf-bc05-7cc2c0b3cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:07.110898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182acb9-28e0-4edf-bc05-7cc2c0b3cc18 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:07.111591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182acb9-28e0-4edf-bc05-7cc2c0b3cc18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:07.113045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182acb9-28e0-4edf-bc05-7cc2c0b3cc18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:07.119600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182acb9-28e0-4edf-bc05-7cc2c0b3cc18 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:07.127600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 131/256] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:07.130975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c8556-7fe4-475e-aa04-595bcddadcd5 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:07.134708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c8556-7fe4-475e-aa04-595bcddadcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:07.135037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c8556-7fe4-475e-aa04-595bcddadcd5 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:07.138113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 132/257] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:07.141628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c8556-7fe4-475e-aa04-595bcddadcd5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:07.170275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb3961-3171-4688-97c7-8dbfc317e42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:07.170948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb3961-3171-4688-97c7-8dbfc317e42a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:07.171241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb3961-3171-4688-97c7-8dbfc317e42a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:07.171515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb3961-3171-4688-97c7-8dbfc317e42a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:07.172343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb3961-3171-4688-97c7-8dbfc317e42a None admin] RBAC: Adding query filter params (role.id=9d4a7bf6cf8243f1abad392422ff53f5, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=11838d29f61a40cc941851ad5a68719f) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:07.172656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb3961-3171-4688-97c7-8dbfc317e42a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:07.174119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb3961-3171-4688-97c7-8dbfc317e42a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:07.181059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 126/258] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:06 2022] GET /identity/v3/role_assignments?user.id=11838d29f61a40cc941851ad5a68719f&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=9d4a7bf6cf8243f1abad392422ff53f5 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:07.182527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e4e60-039a-4fe1-91bb-196b08de9a4c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:07.182821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e4e60-039a-4fe1-91bb-196b08de9a4c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:07.183100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e4e60-039a-4fe1-91bb-196b08de9a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:07.221538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-7e2e4e60-039a-4fe1-91bb-196b08de9a4c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:07.223207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7e2e4e60-039a-4fe1-91bb-196b08de9a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:07.255987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:07.349445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c8556-7fe4-475e-aa04-595bcddadcd5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:07.370884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 127/259] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:07.407253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:07.407944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:07.408226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:07.408501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] PATH_INFO: `/v3/users/1cd348b6a2d4444695294f0bb486d625` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:07.412265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] RBAC: Authorizing `identity:get_user(user_id=1cd348b6a2d4444695294f0bb486d625)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:07.413780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:07.416219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 133/260] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:07 2022] GET /identity/v3/users/1cd348b6a2d4444695294f0bb486d625 => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:07.421856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e4e60-039a-4fe1-91bb-196b08de9a4c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:07.537692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:07.561927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e4e60-039a-4fe1-91bb-196b08de9a4c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:07.604074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:07.604624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:07.604906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:07.605179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:07.691294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:07.691294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:07.791969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:07.810269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 128/261] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:07.816669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:07.879233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:07.949874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bbda-9dd0-4482-90ea-38401e60708e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:07.981043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:07.996391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:07.997062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:07.997346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:07.997626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:08.004307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:08.005820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:08.007509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 134/262] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:07 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:08.013374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:08.063780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1c47f-6996-4d8b-8c84-c523c47528b2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:08.154350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:08.155363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:08.155822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:08.156231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:08.176713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:08.182043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:08.183128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:08.184134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:08.186108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 129/263] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:08.197120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:08.232995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e94fcd-6cfe-46c5-80eb-7e2993f454e5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:08.272528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:08.295984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:08.296753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:08.297034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:08.297310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:08.298062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=1cd348b6a2d4444695294f0bb486d625) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:08.298373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:08.299887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:08.311631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 135/264] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:08 2022] GET /identity/v3/role_assignments?user.id=1cd348b6a2d4444695294f0bb486d625&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:08.370403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d491483-b239-4ba3-9d8a-2726e4a93597 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:08.406976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:08.407525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:08.407823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:08.408090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:08.408768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:08.409067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:08.410494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:08.420632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 130/265] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:08.429287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:08.526783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:08.588653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb9b38-a049-4fab-8d41-bd1b1bfe661b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:08.632019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:08.632671 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:08.632950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:08.633217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] PATH_INFO: `/v3/users/1cd348b6a2d4444695294f0bb486d625` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:08.634740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] RBAC: Authorizing `identity:get_user(user_id=1cd348b6a2d4444695294f0bb486d625)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:08.635916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:08.640649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 136/266] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:08 2022] GET /identity/v3/users/1cd348b6a2d4444695294f0bb486d625 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:08.644058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:08.757036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4d1b0-b706-40ce-9504-5d74ae75b8a4 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:08.760448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:08.814424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137e9c2-0bb4-4030-8a69-d44450f2dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:08.815068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137e9c2-0bb4-4030-8a69-d44450f2dbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:08.815457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137e9c2-0bb4-4030-8a69-d44450f2dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:08.815788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137e9c2-0bb4-4030-8a69-d44450f2dbe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:08.933791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137e9c2-0bb4-4030-8a69-d44450f2dbe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:08.936651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137e9c2-0bb4-4030-8a69-d44450f2dbe4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:09.129304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 131/267] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:09.138574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:09.262850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eacea-cf70-4e34-8bc2-b8194b172110 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.263129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eacea-cf70-4e34-8bc2-b8194b172110 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.263533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eacea-cf70-4e34-8bc2-b8194b172110 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.265327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 132/268] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:25:09.273245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625eec8-407e-40c0-a718-6c4d6d3c00a6 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.273809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625eec8-407e-40c0-a718-6c4d6d3c00a6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.273914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625eec8-407e-40c0-a718-6c4d6d3c00a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.282509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:09.301070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c82c8-3f6a-4939-bcc7-a53a51115323 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:09.314297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3625eec8-407e-40c0-a718-6c4d6d3c00a6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:09.315842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3625eec8-407e-40c0-a718-6c4d6d3c00a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:09.363625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:09.381069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:09.386827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.387387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.387936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.396419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:09.398202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:09.400633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 137/269] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:09 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:09.406677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:09.420095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 133/270] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:09.513992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511dab3a-a8e3-4b1a-bd0b-813f7f009d87 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.514351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511dab3a-a8e3-4b1a-bd0b-813f7f009d87 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.514577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511dab3a-a8e3-4b1a-bd0b-813f7f009d87 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.516230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 134/271] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:25:09.521216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c461b-2f65-4d12-87ae-c165a8e72c92 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.521499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c461b-2f65-4d12-87ae-c165a8e72c92 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.521774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c461b-2f65-4d12-87ae-c165a8e72c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.541884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ee0c461b-2f65-4d12-87ae-c165a8e72c92 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:09.543095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ee0c461b-2f65-4d12-87ae-c165a8e72c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:09.561985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad78a09-91b0-49cf-9dac-f940b3e3f1d4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:09.607914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b62a20-8ed2-406a-bdaf-288564c3e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:09.608536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62a20-8ed2-406a-bdaf-288564c3e05b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.608816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62a20-8ed2-406a-bdaf-288564c3e05b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.609089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62a20-8ed2-406a-bdaf-288564c3e05b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.609830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b62a20-8ed2-406a-bdaf-288564c3e05b None admin] RBAC: Adding query filter params (role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=1cd348b6a2d4444695294f0bb486d625) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:09.610126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b62a20-8ed2-406a-bdaf-288564c3e05b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:09.611789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b62a20-8ed2-406a-bdaf-288564c3e05b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:09.617828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 138/272] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:09 2022] GET /identity/v3/role_assignments?user.id=1cd348b6a2d4444695294f0bb486d625&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:09.621910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20be63-4835-4865-a802-995da127aff2 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.621910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20be63-4835-4865-a802-995da127aff2 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.621910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20be63-4835-4865-a802-995da127aff2 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.621910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 139/273] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:09.639153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 135/274] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:09.641419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57751145-0077-4178-8843-e662348858df None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.641854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57751145-0077-4178-8843-e662348858df None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.642252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57751145-0077-4178-8843-e662348858df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.662771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-57751145-0077-4178-8843-e662348858df None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:09.664368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-57751145-0077-4178-8843-e662348858df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:09.690934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c461b-2f65-4d12-87ae-c165a8e72c92 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:09.785594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:09.789173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:09.814915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c461b-2f65-4d12-87ae-c165a8e72c92 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:09.847894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 140/275] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:09.865647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:09.866304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:09.866577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:09.866845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:09.877849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:09.882608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:09.883465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:09.884339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:09.891264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57751145-0077-4178-8843-e662348858df None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:09.893854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 136/276] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:09.969548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.006427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57751145-0077-4178-8843-e662348858df None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.048698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108fcf7-1c2b-492f-87f5-3851578d88f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.050370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108fcf7-1c2b-492f-87f5-3851578d88f2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.050644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108fcf7-1c2b-492f-87f5-3851578d88f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.050912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108fcf7-1c2b-492f-87f5-3851578d88f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.051690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108fcf7-1c2b-492f-87f5-3851578d88f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:10.052277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108fcf7-1c2b-492f-87f5-3851578d88f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.053844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108fcf7-1c2b-492f-87f5-3851578d88f2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.065713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 141/277] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:10.067350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6442-fb5f-4346-8ce8-9f92b8fb885d None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.067660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6442-fb5f-4346-8ce8-9f92b8fb885d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.067939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6442-fb5f-4346-8ce8-9f92b8fb885d None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.069300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 142/278] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:10.074515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb6442-fb5f-4346-8ce8-9f92b8fb885d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.100649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601853bd-ce20-42a6-ab0e-63db9ac462d0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.130220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.136527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.145475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fd5c3-4fec-4b7c-9435-b7e44ac497e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.146026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd5c3-4fec-4b7c-9435-b7e44ac497e5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.146306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd5c3-4fec-4b7c-9435-b7e44ac497e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.146579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd5c3-4fec-4b7c-9435-b7e44ac497e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.158166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fd5c3-4fec-4b7c-9435-b7e44ac497e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d30fd5c3-4fec-4b7c-9435-b7e44ac497e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:10.163332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:10.164247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 137/279] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:10.170204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a369-4f0b-4bbb-a821-ef3f03134be4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.170934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a369-4f0b-4bbb-a821-ef3f03134be4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.170934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a369-4f0b-4bbb-a821-ef3f03134be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.188233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb6442-fb5f-4346-8ce8-9f92b8fb885d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.188556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-bcb9a369-4f0b-4bbb-a821-ef3f03134be4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:10.189752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bcb9a369-4f0b-4bbb-a821-ef3f03134be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:10.221256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.221883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.222158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.222424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] PATH_INFO: `/v3/users/5dbfb796474c43adbeac0ff1cbd40de9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.235873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] RBAC: Authorizing `identity:get_user(user_id=5dbfb796474c43adbeac0ff1cbd40de9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.237330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.239039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 143/280] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/users/5dbfb796474c43adbeac0ff1cbd40de9 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:10.239787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.259786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 138/281] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:10.260661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a369-4f0b-4bbb-a821-ef3f03134be4 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.342481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6111b60-4529-49d6-9380-9d86d45959e9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.353926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.376684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a369-4f0b-4bbb-a821-ef3f03134be4 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.378727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.379338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.379640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.379931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.380607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:10.380908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.382318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.390980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 144/282] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:10.395672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.397668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.417982 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.418618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.418889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.419157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.420342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.421818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.423428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 139/283] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:10.424222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.518054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da809d9b-2a44-493f-ac0c-7ad701cc5680 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.541827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc469564-3d93-4f35-a8ed-bc36ccf531ae None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.550913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.551518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.551810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.552094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.552815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, user.id=5dbfb796474c43adbeac0ff1cbd40de9, scope.system=all) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:10.553121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.554565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.559722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 145/284] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/role_assignments?user.id=5dbfb796474c43adbeac0ff1cbd40de9&scope.system=all&role.id=69140820db3a4dfea9428bdde1203a39 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:10.560471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.588773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.589709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.590047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.590319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] PATH_INFO: `/v3/users/65c516402d8f4077a809716307fa9d2d` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.592519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] RBAC: Authorizing `identity:get_user(user_id=65c516402d8f4077a809716307fa9d2d)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.594421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.596888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 140/285] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/users/65c516402d8f4077a809716307fa9d2d => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:10.598346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.604034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.657646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.677523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a9f5-bee1-4fae-ba9b-9195ac2b03f5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.718788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01721584-07b4-42a2-8085-b347d24d522d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.720052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.720685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.720964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.722019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.722019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.723716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.748539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 146/286] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:10 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:10.749383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.771400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.773311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.773870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.774813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.792665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:10.806105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:10.807083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:10.807893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:10.814024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 141/287] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:10.820514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:10.862839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.873416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ee599-3bb4-4e2a-b675-16043972b9b1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.916179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.916179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.916179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.916179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.938391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220824c-12d4-4ae3-87e7-e796eef84122 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:10.954542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:10.986949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:10.988273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:10.988743 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:10.989651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:10.990331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:10.990632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.991273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:10.992106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:10.992776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:11.001252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 142/288] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:11.009664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:11.122818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:11.137961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 147/289] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:11.146653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8d9a2-4ea7-46c0-9ff9-ac29669e7497 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:11.147003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:11.193654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:11.194679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:11.195144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:11.195685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] PATH_INFO: `/v3/users/11838d29f61a40cc941851ad5a68719f` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:11.197556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] RBAC: Authorizing `identity:get_user(user_id=11838d29f61a40cc941851ad5a68719f)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:11.199957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:11.202750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 143/290] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:11 2022] GET /identity/v3/users/11838d29f61a40cc941851ad5a68719f => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:11.208141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:11.256344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:11.297270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21517e-2317-4026-9438-ad3387971dcf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:11.357649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a68ca-342e-41a4-bffa-77919229be1a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:11.363384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:11.364064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:11.364344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:11.364616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:11.365749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:11.367228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:11.369482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 148/291] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:11 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:11.376591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:11.420236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:11.420925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:11.421170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:11.421444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:11.423784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:11.425274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:11.429939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 144/292] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:11 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:11.436543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:11.488468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:11.545416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cdb46-6623-4651-9cd3-1e0811014c31 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:11.584919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188be15e-95c9-42ae-ae4c-4143cf92c0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:11.585547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188be15e-95c9-42ae-ae4c-4143cf92c0cc None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:11.585828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188be15e-95c9-42ae-ae4c-4143cf92c0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:11.586099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188be15e-95c9-42ae-ae4c-4143cf92c0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:11.586959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188be15e-95c9-42ae-ae4c-4143cf92c0cc None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=65c516402d8f4077a809716307fa9d2d) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:11.587367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188be15e-95c9-42ae-ae4c-4143cf92c0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:11.589854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188be15e-95c9-42ae-ae4c-4143cf92c0cc None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:11.604110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 149/293] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:11 2022] GET /identity/v3/role_assignments?user.id=65c516402d8f4077a809716307fa9d2d&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=69140820db3a4dfea9428bdde1203a39 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:11.640198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cd19-9164-48b4-8a73-1350a1a7eb17 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:11.676004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:11.699989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8af83b-390f-4c69-9df1-8f586a9ac5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:11.700698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8af83b-390f-4c69-9df1-8f586a9ac5da None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:11.700980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8af83b-390f-4c69-9df1-8f586a9ac5da None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:11.701296 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8af83b-390f-4c69-9df1-8f586a9ac5da None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1/users/11838d29f61a40cc941851ad5a68719f/roles/9d4a7bf6cf8243f1abad392422ff53f5` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:11.703498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8af83b-390f-4c69-9df1-8f586a9ac5da None admin] RBAC: Authorizing `identity:create_grant(project_id=f33b9d002631429689e361f259c154d1, user_id=11838d29f61a40cc941851ad5a68719f, role_id=9d4a7bf6cf8243f1abad392422ff53f5)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:11.705020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8af83b-390f-4c69-9df1-8f586a9ac5da None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:11.731097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 145/294] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:11 2022] PUT /identity/v3/projects/f33b9d002631429689e361f259c154d1/users/11838d29f61a40cc941851ad5a68719f/roles/9d4a7bf6cf8243f1abad392422ff53f5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:12.597279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc84e-34a7-4d52-9f49-c97cd491d3df None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:12.598176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc84e-34a7-4d52-9f49-c97cd491d3df None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:12.598464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc84e-34a7-4d52-9f49-c97cd491d3df None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:12.600219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 150/295] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:12.605727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263ee03-8680-47ca-ac5f-9f064eb256e2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:12.606151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263ee03-8680-47ca-ac5f-9f064eb256e2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:12.606327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263ee03-8680-47ca-ac5f-9f064eb256e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:12.624652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c263ee03-8680-47ca-ac5f-9f064eb256e2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:12.625840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c263ee03-8680-47ca-ac5f-9f064eb256e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:12.694605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 146/296] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:12.960786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffc84e-34a7-4d52-9f49-c97cd491d3df None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.027987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a148c-b94d-4ea9-b90d-1c8359dcac76 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.028267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a148c-b94d-4ea9-b90d-1c8359dcac76 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.028787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a148c-b94d-4ea9-b90d-1c8359dcac76 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.030214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 147/297] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:13.036417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3b4a4-8b41-448e-9494-bef76b1cfbf1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.036696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3b4a4-8b41-448e-9494-bef76b1cfbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.036963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3b4a4-8b41-448e-9494-bef76b1cfbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.057806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a6c3b4a4-8b41-448e-9494-bef76b1cfbf1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:13.059916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a6c3b4a4-8b41-448e-9494-bef76b1cfbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:13.090312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffc84e-34a7-4d52-9f49-c97cd491d3df None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.120526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:13.133179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 148/298] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:13.133788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.134489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.134772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.135040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.147812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2e574-4557-4c55-a3e8-3144337a2451 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.148360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2e574-4557-4c55-a3e8-3144337a2451 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.148830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2e574-4557-4c55-a3e8-3144337a2451 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.150207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:13.150873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 149/299] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:13.155313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:13.156207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:13.156977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 151/300] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:13.159180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b262f-5785-418f-862b-f7e18bf74ad3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.159699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b262f-5785-418f-862b-f7e18bf74ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.160172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b262f-5785-418f-862b-f7e18bf74ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.162121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.179289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-392b262f-5785-418f-862b-f7e18bf74ad3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:13.180974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-392b262f-5785-418f-862b-f7e18bf74ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:13.209468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:13.271599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 150/301] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:13.280048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90894f-84cb-4198-b866-0156dc570876 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.313558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.314180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.314460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.314732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.315423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:13.315787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:13.317476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:13.326261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 152/302] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:13.331944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b262f-5785-418f-862b-f7e18bf74ad3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.404989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.437879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b262f-5785-418f-862b-f7e18bf74ad3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.460181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:13.479894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.480796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.481078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.481354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] PATH_INFO: `/v3/users/5dbfb796474c43adbeac0ff1cbd40de9` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.482493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] RBAC: Authorizing `identity:get_user(user_id=5dbfb796474c43adbeac0ff1cbd40de9)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:13.484798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:13.488091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 151/303] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:13 2022] GET /identity/v3/users/5dbfb796474c43adbeac0ff1cbd40de9 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:13.493218 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.523763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e4882-41e8-4cf5-8f72-2c58f013aa5b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.527130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:13.561011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.561701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.561983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.562256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.573345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:13.578516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:13.579375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:13.580510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:13.583733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 153/304] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:13.584579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.609263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a60b9-8e49-41be-902b-e2fd8f5ecf80 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.643745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.644688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.645156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] PATH_INFO: `/v3/system/users/5dbfb796474c43adbeac0ff1cbd40de9/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.646454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5dbfb796474c43adbeac0ff1cbd40de9, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:13.648071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:13.658039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 152/305] 10.222.0.18 () {58 vars in 1334 bytes} [Mon Aug 22 05:25:13 2022] PUT /identity/v3/system/users/5dbfb796474c43adbeac0ff1cbd40de9/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:13.659417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.696563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055f2e5-b81b-448d-8dad-b464f553760a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.706181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:13.740045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.740851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.741201 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.741545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.742829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:13.744649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:13.751116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 154/306] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:13.752431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.816619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:13.833028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52965a28-d9a6-43f2-883c-986d2af4fad5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.873279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3396f-7612-41de-8d2e-92df284a4323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.873910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3396f-7612-41de-8d2e-92df284a4323 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.874210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3396f-7612-41de-8d2e-92df284a4323 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.874467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3396f-7612-41de-8d2e-92df284a4323 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.875139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3396f-7612-41de-8d2e-92df284a4323 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:13.875440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3396f-7612-41de-8d2e-92df284a4323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:13.876815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad55004-11fd-4d3c-9d70-750031140da9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:13.876931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3396f-7612-41de-8d2e-92df284a4323 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:13.885402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 153/307] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:13.886623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0282f-10c7-4ae1-bf56-2f53fa39908c None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.886907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0282f-10c7-4ae1-bf56-2f53fa39908c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.887184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0282f-10c7-4ae1-bf56-2f53fa39908c None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.890814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 154/308] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:13.892478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0282f-10c7-4ae1-bf56-2f53fa39908c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:13.924230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34225885-05db-4f41-99a9-5046e288c4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:13.924850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225885-05db-4f41-99a9-5046e288c4e9 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.925128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225885-05db-4f41-99a9-5046e288c4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.925634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34225885-05db-4f41-99a9-5046e288c4e9 None admin] PATH_INFO: `/v3/groups` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.926024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225885-05db-4f41-99a9-5046e288c4e9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:13.927490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34225885-05db-4f41-99a9-5046e288c4e9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:13.944438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 155/309] 10.222.0.18 () {60 vars in 1134 bytes} [Mon Aug 22 05:25:13 2022] POST /identity/v3/groups => generated 229 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:13.946072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d09fb-1b1b-4203-a60e-8016dcfd1cad None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:13.946428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d09fb-1b1b-4203-a60e-8016dcfd1cad None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:13.947070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d09fb-1b1b-4203-a60e-8016dcfd1cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:13.967955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-fe8d09fb-1b1b-4203-a60e-8016dcfd1cad None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:13.969605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fe8d09fb-1b1b-4203-a60e-8016dcfd1cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:14.004513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:14.038434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0282f-10c7-4ae1-bf56-2f53fa39908c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:14.053871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 156/310] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:14.088431 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:14.089098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:14.089376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:14.089650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] PATH_INFO: `/v3/users/5e4fb342a258420896f6e60db0ac40dd` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:14.105618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] RBAC: Authorizing `identity:get_user(user_id=5e4fb342a258420896f6e60db0ac40dd)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:14.107110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:14.113473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 155/311] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:13 2022] GET /identity/v3/users/5e4fb342a258420896f6e60db0ac40dd => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:14.115215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d09fb-1b1b-4203-a60e-8016dcfd1cad None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:14.229390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d09fb-1b1b-4203-a60e-8016dcfd1cad None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:14.267466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:14.268462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:14.268957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:14.269420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:14.270363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] RBAC: Adding query filter params (role.id=9d4a7bf6cf8243f1abad392422ff53f5, user.id=5e4fb342a258420896f6e60db0ac40dd, scope.system=all) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:14.270863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:14.272540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:14.278604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 157/312] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 22 05:25:14 2022] GET /identity/v3/role_assignments?user.id=5e4fb342a258420896f6e60db0ac40dd&scope.system=all&role.id=9d4a7bf6cf8243f1abad392422ff53f5 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:14.320451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:14.333749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:14.431954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8644f-af6e-42ec-a160-25eac6af726e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:14.469734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:14.470726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:14.471242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:14.471732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:14.483074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:14.488148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:14.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:14.489795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:14.492382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 156/313] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:14.498506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:14.532425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:14.608398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bad761-05cd-4007-9263-7f55b9f567b0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:14.642478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:14.643092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:14.643363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:14.643651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:14.644324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:14.644615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:14.645989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:14.654985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 158/314] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:14.660620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:14.734638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:14.768360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45bbb2-3b00-45b0-a7e3-6ef3e4f3cad5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:14.807606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:14.808290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:14.808569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:14.808842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] PATH_INFO: `/v3/users/11838d29f61a40cc941851ad5a68719f` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:14.810040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] RBAC: Authorizing `identity:get_user(user_id=11838d29f61a40cc941851ad5a68719f)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:14.811520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:14.813586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 157/315] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:14 2022] GET /identity/v3/users/11838d29f61a40cc941851ad5a68719f => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:14.821113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:14.916709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:14.926571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fa9f4-8f51-4aeb-bc19-388905d0b1df None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:14.964520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:14.965190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:14.965466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:14.965730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:15.032715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:15.034125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:15.123061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:15.133195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 159/316] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:15.139240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:15.247280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635232a-f7e3-492b-988a-eb90f1fd18af None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:15.282257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb9bb0-b961-4e13-bcf0-792b50739ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:15.282965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9bb0-b961-4e13-bcf0-792b50739ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:15.283189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9bb0-b961-4e13-bcf0-792b50739ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:15.283462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9bb0-b961-4e13-bcf0-792b50739ec7 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:15.284625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb9bb0-b961-4e13-bcf0-792b50739ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:15.286151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb9bb0-b961-4e13-bcf0-792b50739ec7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:15.288154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 158/317] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:15 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:15.293756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:15.400017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d5fd-da03-4a2c-9e0a-92de77493be7 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:15.437681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fd2f9-030d-4ce3-a7f9-f7e637434e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:15.438693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fd2f9-030d-4ce3-a7f9-f7e637434e8f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:15.439157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fd2f9-030d-4ce3-a7f9-f7e637434e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:15.439644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fd2f9-030d-4ce3-a7f9-f7e637434e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:15.440628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fd2f9-030d-4ce3-a7f9-f7e637434e8f None admin] RBAC: Adding query filter params (role.id=9d4a7bf6cf8243f1abad392422ff53f5, scope.project.id=f33b9d002631429689e361f259c154d1, user.id=11838d29f61a40cc941851ad5a68719f) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:15.441140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fd2f9-030d-4ce3-a7f9-f7e637434e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:15.442794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fd2f9-030d-4ce3-a7f9-f7e637434e8f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:15.450135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 160/318] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:15 2022] GET /identity/v3/role_assignments?user.id=11838d29f61a40cc941851ad5a68719f&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=9d4a7bf6cf8243f1abad392422ff53f5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:15.694418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587788-7ab8-477b-9f50-bdd501272231 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:15.694935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587788-7ab8-477b-9f50-bdd501272231 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:15.695009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587788-7ab8-477b-9f50-bdd501272231 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:15.696535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 159/319] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:15.701736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee2a05-ecb7-4387-a939-c78a1f5363f7 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:15.702030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee2a05-ecb7-4387-a939-c78a1f5363f7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:15.702301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee2a05-ecb7-4387-a939-c78a1f5363f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:15.720532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-adee2a05-ecb7-4387-a939-c78a1f5363f7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:15.721732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-adee2a05-ecb7-4387-a939-c78a1f5363f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:15.777775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:15.797030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 161/320] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:16.021981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e344c7-1caf-4355-98fa-c27711b4046b None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.022164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e344c7-1caf-4355-98fa-c27711b4046b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.022400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e344c7-1caf-4355-98fa-c27711b4046b None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.023952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 160/321] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:16.029139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a23c2a-3599-4409-9754-fc914d9a6743 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.029602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a23c2a-3599-4409-9754-fc914d9a6743 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.029697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a23c2a-3599-4409-9754-fc914d9a6743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.048173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-02a23c2a-3599-4409-9754-fc914d9a6743 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:16.049339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-02a23c2a-3599-4409-9754-fc914d9a6743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:16.066259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e344c7-1caf-4355-98fa-c27711b4046b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:16.079199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:16.121143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 162/322] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:16.178844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e344c7-1caf-4355-98fa-c27711b4046b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:16.215308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bb874-dc42-4b21-9d99-ae232d0afc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:16.216347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bb874-dc42-4b21-9d99-ae232d0afc43 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.216830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bb874-dc42-4b21-9d99-ae232d0afc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.217333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bb874-dc42-4b21-9d99-ae232d0afc43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.228626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bb874-dc42-4b21-9d99-ae232d0afc43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-998bb874-dc42-4b21-9d99-ae232d0afc43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:16.233634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:16.234487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:16.235307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:16.237709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 161/323] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:16.243669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a23c2a-3599-4409-9754-fc914d9a6743 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:16.268610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8281556-9250-42fc-9cc6-7a064aa74e95 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.268948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8281556-9250-42fc-9cc6-7a064aa74e95 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.269316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8281556-9250-42fc-9cc6-7a064aa74e95 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.271393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 162/324] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:16.277352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3022-f0b4-412c-9499-5f16f9fbacbf None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.277578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3022-f0b4-412c-9499-5f16f9fbacbf None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.277837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3022-f0b4-412c-9499-5f16f9fbacbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.302122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3fbb3022-f0b4-412c-9499-5f16f9fbacbf None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:16.304025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3fbb3022-f0b4-412c-9499-5f16f9fbacbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:16.347263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a23c2a-3599-4409-9754-fc914d9a6743 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:16.363351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:16.378550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:16.386370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 163/325] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:16.387240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb3022-f0b4-412c-9499-5f16f9fbacbf None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:16.388399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:16.389024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.389303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.389573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.390261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:16.390554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:16.391979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:16.403156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 163/326] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:16.408567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:16.495588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb3022-f0b4-412c-9499-5f16f9fbacbf None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:16.513783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc073f0-a553-4aef-98a4-2a1b0ac9f8a8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:16.530718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:16.531392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.531694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.531961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.533061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:16.534605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:16.541009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 164/327] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:16.541831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:16.547862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:16.548555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.548836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.549134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] PATH_INFO: `/v3/users/5dbfb796474c43adbeac0ff1cbd40de9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.550260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] RBAC: Authorizing `identity:get_user(user_id=5dbfb796474c43adbeac0ff1cbd40de9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:16.551757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:16.553520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 164/328] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:16 2022] GET /identity/v3/users/5dbfb796474c43adbeac0ff1cbd40de9 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:16.559143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:16.608888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:16.610870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:16.651857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe5794-82f7-490f-a86c-23944bcda6f1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:16.667173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cba2a1-3594-48fe-9a65-d34932e28b91 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:16.686134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:16.686754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.687030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.687304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] PATH_INFO: `/v3/groups` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.687955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:16.689397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:16.701843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:16.702545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.702826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.703096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.703859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] RBAC: Adding query filter params (role.id=69140820db3a4dfea9428bdde1203a39, user.id=5dbfb796474c43adbeac0ff1cbd40de9, scope.system=all) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:16.704164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:16.705607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:16.706135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 165/329] 10.222.0.18 () {60 vars in 1134 bytes} [Mon Aug 22 05:25:16 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:16.706946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:16.735610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 165/330] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 22 05:25:16 2022] GET /identity/v3/role_assignments?user.id=5dbfb796474c43adbeac0ff1cbd40de9&scope.system=all&role.id=69140820db3a4dfea9428bdde1203a39 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:16.837646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802392b2-1d8f-4df7-be05-f3d21e1336b2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:16.866450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:16.881203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:16.881870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:16.882178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:16.882445 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:16.893939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:16.899674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:16.900710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:16.901500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 166/331] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:16.906804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:17.016465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e5d5-cc53-482e-9242-4a63539490ef None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:17.018045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:17.052474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:17.053095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:17.053357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:17.053632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:17.054303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:17.054609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:17.056358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:17.065145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 166/332] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:17.070605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:17.175482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a39ca0-892c-4eb4-a551-c0d582b34894 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:17.211153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db152b-7479-4bb4-a5bc-b531b908a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:17.212118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db152b-7479-4bb4-a5bc-b531b908a14b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:17.212530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db152b-7479-4bb4-a5bc-b531b908a14b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:17.212934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db152b-7479-4bb4-a5bc-b531b908a14b None admin] PATH_INFO: `/v3/users/5e4fb342a258420896f6e60db0ac40dd` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:17.214228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db152b-7479-4bb4-a5bc-b531b908a14b None admin] RBAC: Authorizing `identity:get_user(user_id=5e4fb342a258420896f6e60db0ac40dd)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:17.215858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db152b-7479-4bb4-a5bc-b531b908a14b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:17.217993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 167/333] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:17 2022] GET /identity/v3/users/5e4fb342a258420896f6e60db0ac40dd => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:17.226519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:17.334053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b2ed1-e324-494b-a287-f490ef606ac4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:17.369090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a6923-0c5c-43cc-8d83-5038f6d912f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:17.370091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a6923-0c5c-43cc-8d83-5038f6d912f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:17.370537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a6923-0c5c-43cc-8d83-5038f6d912f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:17.371000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a6923-0c5c-43cc-8d83-5038f6d912f9 None admin] PATH_INFO: `/v3/system/users/5e4fb342a258420896f6e60db0ac40dd/roles/9d4a7bf6cf8243f1abad392422ff53f5` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:17.372699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a6923-0c5c-43cc-8d83-5038f6d912f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5e4fb342a258420896f6e60db0ac40dd, role_id=9d4a7bf6cf8243f1abad392422ff53f5)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:17.374254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a6923-0c5c-43cc-8d83-5038f6d912f9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:17.385334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 167/334] 10.222.0.18 () {58 vars in 1334 bytes} [Mon Aug 22 05:25:17 2022] PUT /identity/v3/system/users/5e4fb342a258420896f6e60db0ac40dd/roles/9d4a7bf6cf8243f1abad392422ff53f5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:18.944240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ed3f-c29f-48be-95e8-f1ea2427175a None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:18.944780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ed3f-c29f-48be-95e8-f1ea2427175a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:18.945011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ed3f-c29f-48be-95e8-f1ea2427175a None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:18.946621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 168/335] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:18.952779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40531a9-7a82-462a-93c3-d5a125029e2f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:18.953198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40531a9-7a82-462a-93c3-d5a125029e2f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:18.953330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40531a9-7a82-462a-93c3-d5a125029e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:18.974860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e40531a9-7a82-462a-93c3-d5a125029e2f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:18.976351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e40531a9-7a82-462a-93c3-d5a125029e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:19.011247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:19.069322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 168/336] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:19.117999 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e11e7-3730-45ef-8413-a6433c6d234e None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:19.118282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e11e7-3730-45ef-8413-a6433c6d234e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:19.118717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e11e7-3730-45ef-8413-a6433c6d234e None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:19.120514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 169/337] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:19.125632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbedb98-4e7c-4d12-8e10-9c6bf3c0bd58 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:19.126082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbedb98-4e7c-4d12-8e10-9c6bf3c0bd58 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:19.126517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbedb98-4e7c-4d12-8e10-9c6bf3c0bd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:19.150779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5dbedb98-4e7c-4d12-8e10-9c6bf3c0bd58 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:19.152229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5dbedb98-4e7c-4d12-8e10-9c6bf3c0bd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:19.246145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 169/338] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:19.454639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e11e7-3730-45ef-8413-a6433c6d234e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:19.519856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:19.575739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e11e7-3730-45ef-8413-a6433c6d234e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:19.622083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:19.623132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:19.623669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:19.624153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:19.635780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:19.641082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:19.641944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:19.642829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:19.645442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 170/339] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:19.651061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbedb98-4e7c-4d12-8e10-9c6bf3c0bd58 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:19.763158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:19.777727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbedb98-4e7c-4d12-8e10-9c6bf3c0bd58 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:19.816210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:19.817178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:19.817664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:19.818135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:19.818994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:19.819480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:19.821155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:19.833906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 170/340] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:19.835207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:19.848154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:19.993910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea9a53-d622-4f72-9f48-50f5f7bc3676 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:20.014286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:20.046401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c9e45-3109-496d-9364-7f53843797b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:20.047061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c9e45-3109-496d-9364-7f53843797b1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.047344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c9e45-3109-496d-9364-7f53843797b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.049026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c9e45-3109-496d-9364-7f53843797b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.061914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c9e45-3109-496d-9364-7f53843797b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:20.065581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4044e1b-af46-435c-9260-d058f61ba542 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-974c9e45-3109-496d-9364-7f53843797b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:20.067963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:20.068860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:20.069675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:20.072149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 171/341] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:20.073685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5268-d276-4477-83ba-27eb7f3ca5c4 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.074144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5268-d276-4477-83ba-27eb7f3ca5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.074608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5268-d276-4477-83ba-27eb7f3ca5c4 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.076407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 172/342] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:20.077826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de5268-d276-4477-83ba-27eb7f3ca5c4 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:20.140309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848281f-2eda-494a-a34a-4d7d8a0d6c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:20.141041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848281f-2eda-494a-a34a-4d7d8a0d6c96 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.141410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848281f-2eda-494a-a34a-4d7d8a0d6c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.141758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848281f-2eda-494a-a34a-4d7d8a0d6c96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.174516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:20.209413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de5268-d276-4477-83ba-27eb7f3ca5c4 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:20.228928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848281f-2eda-494a-a34a-4d7d8a0d6c96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:20.228928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848281f-2eda-494a-a34a-4d7d8a0d6c96 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:20.257007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f16c65-07f6-4592-bb63-32aeac7d463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:20.258117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f16c65-07f6-4592-bb63-32aeac7d463c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.258800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f16c65-07f6-4592-bb63-32aeac7d463c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.259498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f16c65-07f6-4592-bb63-32aeac7d463c None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.260492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f16c65-07f6-4592-bb63-32aeac7d463c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:20.261062 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f16c65-07f6-4592-bb63-32aeac7d463c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:20.264090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f16c65-07f6-4592-bb63-32aeac7d463c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:20.274652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 173/343] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:20.276434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102899c0-6271-41c7-a4f0-e30cded9b7bb None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.276943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102899c0-6271-41c7-a4f0-e30cded9b7bb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.277410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102899c0-6271-41c7-a4f0-e30cded9b7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.298201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-102899c0-6271-41c7-a4f0-e30cded9b7bb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:20.299906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-102899c0-6271-41c7-a4f0-e30cded9b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:20.332998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:20.334011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 171/344] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:20.340108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30399-ce58-4966-b4ce-e096532c9e3e None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.340453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30399-ce58-4966-b4ce-e096532c9e3e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.340720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30399-ce58-4966-b4ce-e096532c9e3e None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.342474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 172/345] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:20.343248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30399-ce58-4966-b4ce-e096532c9e3e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:20.380867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 174/346] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:20.382939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4bb7-14a0-4806-84d0-e84fabd43fc5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.383737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4bb7-14a0-4806-84d0-e84fabd43fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.384329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4bb7-14a0-4806-84d0-e84fabd43fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.402767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-1fee4bb7-14a0-4806-84d0-e84fabd43fc5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:20.404435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1fee4bb7-14a0-4806-84d0-e84fabd43fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:20.422277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:20.481800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 175/347] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:20.483261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4bb7-14a0-4806-84d0-e84fabd43fc5 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:20.505238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30399-ce58-4966-b4ce-e096532c9e3e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:20.584397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:20.584958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.585230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.585503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.629176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4bb7-14a0-4806-84d0-e84fabd43fc5 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:20.652025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:20.679553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:20.680647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.681204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.681430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.683804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:20.685659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:20.686478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:20.687991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:20.688099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 176/348] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:20 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:20.695100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:20.760188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:20.838916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1857b-1ea1-46d5-8e34-e031694545d6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:20.840594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 173/349] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:20.845985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:20.890173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222b419-8147-4bc6-b92e-d4059dd0e50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:20.890974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222b419-8147-4bc6-b92e-d4059dd0e50c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.891317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222b419-8147-4bc6-b92e-d4059dd0e50c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.891616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222b419-8147-4bc6-b92e-d4059dd0e50c None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.904327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222b419-8147-4bc6-b92e-d4059dd0e50c None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:20.905908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222b419-8147-4bc6-b92e-d4059dd0e50c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:20.908142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 177/350] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:20 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:20.909352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0ee46-062f-4b97-a05f-0bdd72ee4f2e None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.909633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0ee46-062f-4b97-a05f-0bdd72ee4f2e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.910005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0ee46-062f-4b97-a05f-0bdd72ee4f2e None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.911300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 178/351] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:20.917118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde3acb-6432-4e2e-a877-d88c57145695 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:20.917403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde3acb-6432-4e2e-a877-d88c57145695 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:20.917678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde3acb-6432-4e2e-a877-d88c57145695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:20.948713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6cde3acb-6432-4e2e-a877-d88c57145695 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:20.950009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6cde3acb-6432-4e2e-a877-d88c57145695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:20.987150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77716332-47b2-43b9-97d9-c2fd663b9df1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:20.987915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.023997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.024995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.025171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.025408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.026559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:21.030005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:21.030005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 174/352] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:20 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:21.030805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.059974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.073866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 179/353] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:21.075339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde3acb-6432-4e2e-a877-d88c57145695 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.182557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d534cc-13ef-4b3a-ae0b-c5531d262b69 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.189983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde3acb-6432-4e2e-a877-d88c57145695 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.219039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.219797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.220141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.220478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.221285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=5331948968af49e2bac622c653dc3e25) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:21.221713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:21.223655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:21.230061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 175/354] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=69140820db3a4dfea9428bdde1203a39 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:21.230797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.231602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.232341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.232726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.233061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.236491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:21.238120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:21.239961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 180/355] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:21.240840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.276001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.331951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.352440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2dc96-348a-4b30-ba0f-75cfd555c042 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.358061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4378b42-93ff-428a-8be7-ccb421178f6b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.391999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.392903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.393286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.393568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.395316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.396217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.396655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.397067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.408148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:21.411418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:21.413599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:21.414780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:21.415654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 176/356] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:21.416331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:21.419075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:21.421122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:21.422093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:21.424673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 181/357] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:21.426301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.534618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.561223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602198-1a60-493b-9f26-ba741821dbca None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.561710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469ad0a-abca-45af-bf3d-9202a9722aa0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.599058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.599969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.600409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ed0da-3e1f-4fa6-b29d-484b77a03bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.600534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.600915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.601019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ed0da-3e1f-4fa6-b29d-484b77a03bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.601293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ed0da-3e1f-4fa6-b29d-484b77a03bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.601566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ed0da-3e1f-4fa6-b29d-484b77a03bb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.602374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ed0da-3e1f-4fa6-b29d-484b77a03bb7 None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=5331948968af49e2bac622c653dc3e25) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:21.602683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ed0da-3e1f-4fa6-b29d-484b77a03bb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:21.604297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ed0da-3e1f-4fa6-b29d-484b77a03bb7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:21.618147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 177/358] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:21.618147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9704c-c01b-414b-b45d-4a527df112b7 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.618147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9704c-c01b-414b-b45d-4a527df112b7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.618147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9704c-c01b-414b-b45d-4a527df112b7 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.618147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 178/359] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:21.618147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9704c-c01b-414b-b45d-4a527df112b7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.628240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.629088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:21.634437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:21.635407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:21.636429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:21.638720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 182/360] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:21.643603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.773522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9704c-c01b-414b-b45d-4a527df112b7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.788345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f694c-e20d-4a67-af7a-31bca74d75bc None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:21.820432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219004b-d05b-43ec-8d34-c087d32e31b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.822032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219004b-d05b-43ec-8d34-c087d32e31b5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.822484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219004b-d05b-43ec-8d34-c087d32e31b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.822897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219004b-d05b-43ec-8d34-c087d32e31b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.823749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219004b-d05b-43ec-8d34-c087d32e31b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:21.823916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219004b-d05b-43ec-8d34-c087d32e31b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:21.825306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219004b-d05b-43ec-8d34-c087d32e31b5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:21.834803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 179/361] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:21.835014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:21.835915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.836180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc393-06d2-44b1-9966-df6bf3c5d378 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:21.836373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.836869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc393-06d2-44b1-9966-df6bf3c5d378 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:21.836963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.837475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc393-06d2-44b1-9966-df6bf3c5d378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:21.837955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:21.838357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:21.840291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:21.855394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 183/362] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:21.856517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.858451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4adbc393-06d2-44b1-9966-df6bf3c5d378 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:21.859773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4adbc393-06d2-44b1-9966-df6bf3c5d378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:21.866924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.961572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:21.971206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 180/363] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:21.972368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc393-06d2-44b1-9966-df6bf3c5d378 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:21.981829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e34e6-ecf2-4931-af57-0cd912d58ff0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.040892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.041454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.041738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.042023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.042696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:22.043002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.044633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.062783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 184/364] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:22.063826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.090844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc393-06d2-44b1-9966-df6bf3c5d378 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.140944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.141887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.142393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.142828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] PATH_INFO: `/v3/users/5e4fb342a258420896f6e60db0ac40dd` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.145045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] RBAC: Authorizing `identity:get_user(user_id=5e4fb342a258420896f6e60db0ac40dd)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.146682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.149055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 181/365] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:25:21 2022] GET /identity/v3/users/5e4fb342a258420896f6e60db0ac40dd => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:22.150303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.192923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:22.241101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:22.262858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5ad4f-2918-43e7-8e26-fc1720601f58 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.273181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127838f-f0d1-4651-8bf5-1c2165b28bef None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.307912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.308450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.308725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.308990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.345572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.346563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.347029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.347849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.390805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.392397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.454610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.456346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.459066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:22.496984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 182/366] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:22.498043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.547970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:22.578039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 185/367] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:22.578978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.603326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2dca-3831-4f97-8e69-3397583665b5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.638911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.639533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.639847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.640114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.641004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] RBAC: Adding query filter params (role.id=9d4a7bf6cf8243f1abad392422ff53f5, user.id=5e4fb342a258420896f6e60db0ac40dd, scope.system=all) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:22.641307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.642811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.657432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 183/368] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/role_assignments?user.id=5e4fb342a258420896f6e60db0ac40dd&scope.system=all&role.id=9d4a7bf6cf8243f1abad392422ff53f5 => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:22.658359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.690173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:22.694961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38038b-aa7e-433d-8520-009d96abc92b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.741749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.742907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.744035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.744462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.756414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:22.762400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:22.763343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:22.764161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:22.771248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 186/369] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:22.772668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.780399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f44d03-8024-425e-bc4a-0d7b08a83a0d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.815457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:22.818485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.819292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.819567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.819863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.821157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.822776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.824673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 184/370] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:22.825394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.892644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177de2a-9488-4c4e-bf15-adcf9c7172da None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.927909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:22.935855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.937192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.937483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.937764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.939199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.940180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d5e7e-5d2a-453b-b48b-df12b8b55185 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:22.940835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.944108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 187/371] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:22.944995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:22.983780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:22.984406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:22.984682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:22.984947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:22.985717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:22.986115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:22.987661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:22.998379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 185/372] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:22.999281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:23.079034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:23.086757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688477-1a27-4ae0-93cd-62dfe3a9bcc9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:23.132809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:23.133702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.134079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.134483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.135761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:23.136035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b4450-a4a6-4408-97b9-8af5ce482745 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:23.137393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:23.140448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 188/373] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:23.144581 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:23.184124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:23.185186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.185683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.186188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.188438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:23.190190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:23.192032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 186/374] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:22 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:23.192981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:23.221913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:23.275979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e5fed-9c2a-430e-965c-6f14ac06bdf6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:23.314533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:23.315683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.316150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.316605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.321538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741f0db-a07e-4bd1-86cd-ad4c8f7a0891 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:23.350459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:23.359098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:23.359773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.360108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.360385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.361131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=f33b9d002631429689e361f259c154d1) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:23.361445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:23.362952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:23.369511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 187/375] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:23 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:23.370439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:23.398070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:23.400026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:23.469234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:23.487499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b436d-93b8-444a-b26c-9755b0fc80c8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:23.512685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 189/376] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:23.518655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:23.530752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:23.531569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.531879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.532241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.533010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=f33b9d002631429689e361f259c154d1) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:23.533324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:23.534889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:23.541960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 188/377] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:23 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=69140820db3a4dfea9428bdde1203a39 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:23.604148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:23.640738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4a53c-2677-4593-8018-b28ce107afb4 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:23.679079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bf580-19cb-4060-91a2-a8d41dbf6dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:23.679691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf580-19cb-4060-91a2-a8d41dbf6dba None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.679975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf580-19cb-4060-91a2-a8d41dbf6dba None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.680373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf580-19cb-4060-91a2-a8d41dbf6dba None admin] PATH_INFO: `/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.681646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bf580-19cb-4060-91a2-a8d41dbf6dba None admin] RBAC: Authorizing `identity:get_project(project_id=fb2b2de175ff44f7b8683cb1604be1f7)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:23.683329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bf580-19cb-4060-91a2-a8d41dbf6dba None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:23.685558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 190/378] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:23 2022] GET /identity/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:23.691108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:23.811144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202aaf8-dd6c-48b3-ba2c-cfa91dd4de0b None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.811445 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202aaf8-dd6c-48b3-ba2c-cfa91dd4de0b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.811757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202aaf8-dd6c-48b3-ba2c-cfa91dd4de0b None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.813506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 191/379] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:25:23.817251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757211e-0ff2-4d01-aced-7d41369254dd None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:23.819245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6d11f-ddaf-433a-ab7d-a97ce040a994 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.819530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6d11f-ddaf-433a-ab7d-a97ce040a994 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.819868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6d11f-ddaf-433a-ab7d-a97ce040a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.840956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-57d6d11f-ddaf-433a-ab7d-a97ce040a994 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:23.842295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-57d6d11f-ddaf-433a-ab7d-a97ce040a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:23.873313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:23.873965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:23.874244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:23.874514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] PATH_INFO: `/v3/groups/c3176c73a4d34f60ac3b5db1cef4556e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:23.883732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] RBAC: Authorizing `identity:get_group(group_id=c3176c73a4d34f60ac3b5db1cef4556e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:23.885734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:23.888169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 189/380] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:23 2022] GET /identity/v3/groups/c3176c73a4d34f60ac3b5db1cef4556e => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:23.903882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:23.924683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:23.942239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 192/381] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:24.018167 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1d5de-0e3f-47e1-92b1-4a1584da52cf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:24.059386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8281f5-2e1f-4fe2-aebd-7656f82476cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:24.060035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8281f5-2e1f-4fe2-aebd-7656f82476cd None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:24.060314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8281f5-2e1f-4fe2-aebd-7656f82476cd None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:24.060583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8281f5-2e1f-4fe2-aebd-7656f82476cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:24.061322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8281f5-2e1f-4fe2-aebd-7656f82476cd None admin] RBAC: Adding query filter params (group.id=c3176c73a4d34f60ac3b5db1cef4556e, role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=fb2b2de175ff44f7b8683cb1604be1f7) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:24.061628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8281f5-2e1f-4fe2-aebd-7656f82476cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:24.063095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8281f5-2e1f-4fe2-aebd-7656f82476cd None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:24.069360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 190/382] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:23 2022] GET /identity/v3/role_assignments?group.id=c3176c73a4d34f60ac3b5db1cef4556e&scope.project.id=fb2b2de175ff44f7b8683cb1604be1f7&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:24.267827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6006b-17c0-4bdd-a7d7-7dbd9a2594ea None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:24.268025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6006b-17c0-4bdd-a7d7-7dbd9a2594ea None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:24.268369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6006b-17c0-4bdd-a7d7-7dbd9a2594ea None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:24.270061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 193/383] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:24.277521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121442d-f664-4a9b-a50f-ba84a76b7d2b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:24.277839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121442d-f664-4a9b-a50f-ba84a76b7d2b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:24.278074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121442d-f664-4a9b-a50f-ba84a76b7d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:24.298561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f121442d-f664-4a9b-a50f-ba84a76b7d2b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:24.299787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f121442d-f664-4a9b-a50f-ba84a76b7d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:24.319317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6006b-17c0-4bdd-a7d7-7dbd9a2594ea None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:24.377365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 191/384] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:24.423976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:24.477080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6006b-17c0-4bdd-a7d7-7dbd9a2594ea None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:24.514627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:24.515334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:24.515699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:24.515973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:24.526567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:24.531244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:24.532209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:24.533021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 194/385] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:24.541620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121442d-f664-4a9b-a50f-ba84a76b7d2b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:24.566265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:24.662075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121442d-f664-4a9b-a50f-ba84a76b7d2b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:24.700854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:24.701885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:24.702358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:24.702819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:24.703759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:24.704278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:24.705933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:24.718732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 192/386] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:24.724729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:24.732696 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:24.819910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:24.859284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752d679-5a5f-4adf-9231-1bcda428f680 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:24.900561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:24.902155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:24.902648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:24.903130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:24.915897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:24.921354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:24.922225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:24.923019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 193/387] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:24.932250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:24.948023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d0402-72d6-4dc3-be74-ada535e6752e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:25.004128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:25.043232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:25.043961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.044247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.044518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.048237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:25.049722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:25.051769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 195/388] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:24 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:25.052533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bd448-1fc0-4e17-90b9-0bb3893d4043 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:25.059666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:25.089797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:25.100352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:25.101048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.101327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.102505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.103956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:25.104270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:25.105757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:25.120815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 194/389] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:25.127639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:25.251755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9478ff-837a-471d-9704-20e2ba6d1272 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:25.292675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:25.293637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.293917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.294190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.295338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:25.297852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:25.299845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 195/390] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:25 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:25.305940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:25.384062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:25.405563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054fd5-d4a5-4da4-b7a6-1a9564f455a5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:25.423415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6efe1d-c274-476f-acf7-cc074272d4d3 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:25.463753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:25.464572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.464944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.465306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.467168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:25.468804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:25.471035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 196/391] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:25 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:25.477037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:25.500843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:25.501518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.501801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.502078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.508331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:25.509842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:25.511967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 196/392] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:25 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:25.517698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:25.584275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e5340-dadf-4791-bb3d-1dac82cd7914 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:25.612407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:25.622107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5394-febf-4a18-a8aa-f7248818c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:25.622852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5394-febf-4a18-a8aa-f7248818c3e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.623131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5394-febf-4a18-a8aa-f7248818c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.623409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5394-febf-4a18-a8aa-f7248818c3e0 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25/groups/906fd3862ffb42ada255181e587e3119/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.625866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5394-febf-4a18-a8aa-f7248818c3e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5331948968af49e2bac622c653dc3e25, group_id=906fd3862ffb42ada255181e587e3119, role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:25.627375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5394-febf-4a18-a8aa-f7248818c3e0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:25.631618 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:25.648160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 197/393] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 22 05:25:25 2022] PUT /identity/v3/projects/5331948968af49e2bac622c653dc3e25/groups/906fd3862ffb42ada255181e587e3119/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:25:25.707708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fddab-e37a-40af-b885-475573c77877 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:25.753470 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf894572-0cbe-46cc-bde9-e3169d20f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:25.754169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf894572-0cbe-46cc-bde9-e3169d20f46e None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.754448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf894572-0cbe-46cc-bde9-e3169d20f46e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.754769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf894572-0cbe-46cc-bde9-e3169d20f46e None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25/groups/906fd3862ffb42ada255181e587e3119/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.757926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf894572-0cbe-46cc-bde9-e3169d20f46e None admin] RBAC: Authorizing `identity:create_grant(project_id=5331948968af49e2bac622c653dc3e25, group_id=906fd3862ffb42ada255181e587e3119, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:25.759701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf894572-0cbe-46cc-bde9-e3169d20f46e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:25.782014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 197/394] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 22 05:25:25 2022] PUT /identity/v3/projects/5331948968af49e2bac622c653dc3e25/groups/906fd3862ffb42ada255181e587e3119/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:25.981268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af6174-229f-47d5-901e-ad7ca606ed47 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.981645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af6174-229f-47d5-901e-ad7ca606ed47 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.981824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af6174-229f-47d5-901e-ad7ca606ed47 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:25.983484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 198/395] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:25.989379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5b07c-39a3-4718-a422-5fe24acde95f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:25.989668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5b07c-39a3-4718-a422-5fe24acde95f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:25.989950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5b07c-39a3-4718-a422-5fe24acde95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:26.013027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a7f5b07c-39a3-4718-a422-5fe24acde95f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:26.014240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a7f5b07c-39a3-4718-a422-5fe24acde95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:26.104490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 198/396] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:26.448583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb931e3-4291-43fa-a22f-2a96f75b7168 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:26.449128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb931e3-4291-43fa-a22f-2a96f75b7168 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:26.449128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb931e3-4291-43fa-a22f-2a96f75b7168 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:26.450730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 199/397] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:26.456232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff12e6-6539-4cc3-b366-d42fe77e8c2e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:26.457054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff12e6-6539-4cc3-b366-d42fe77e8c2e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:26.457493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff12e6-6539-4cc3-b366-d42fe77e8c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:26.477462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-b9ff12e6-6539-4cc3-b366-d42fe77e8c2e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:26.478919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b9ff12e6-6539-4cc3-b366-d42fe77e8c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:26.527823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:26.558385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 199/398] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:26.575825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9484a41-61fd-4b3d-b572-0aae5f990d52 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:26.576393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9484a41-61fd-4b3d-b572-0aae5f990d52 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:26.576849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9484a41-61fd-4b3d-b572-0aae5f990d52 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:26.578601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 200/399] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:26.583680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f742495-360b-4832-8237-1bc2352bb697 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:26.583970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f742495-360b-4832-8237-1bc2352bb697 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:26.584245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f742495-360b-4832-8237-1bc2352bb697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:26.612717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3f742495-360b-4832-8237-1bc2352bb697 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:26.613899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3f742495-360b-4832-8237-1bc2352bb697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:26.658452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9484a41-61fd-4b3d-b572-0aae5f990d52 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:26.705008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 200/400] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:26.781147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9484a41-61fd-4b3d-b572-0aae5f990d52 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:26.824591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:26.825232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:26.825505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:26.825770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:26.840834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:26.845532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:26.846383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:26.847170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 201/401] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:26.855220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f742495-360b-4832-8237-1bc2352bb697 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:26.984167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f742495-360b-4832-8237-1bc2352bb697 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:26.996752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:27.028402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.029049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.029311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.029582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.030259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:27.030559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:27.031961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:27.036635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.041538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 201/402] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:27.047189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.126085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:27.176660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18d545-7160-4ccd-8955-b0259b4aef1e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.197195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c4576-f14f-45c5-beda-ba5db232912c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.221551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.222975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.223251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.223809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.225123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:27.227252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:27.229226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 202/403] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:27.230172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.243558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.244158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.244431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.244702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.262664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:27.269170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:27.270052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:27.270833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 202/404] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:27.272327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.286629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:27.362675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e85d27-a2a2-4c9e-a70d-371440e0065b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.402984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebce3-38b4-4709-8ffe-944122be0543 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.407266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.408509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.408922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.409340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.418200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:27.426466 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:27.432597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:27.433670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:27.434454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:27.438709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 203/405] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:27.439610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.455778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.457132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.457921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.458348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.459864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:27.462341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:27.464166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 203/406] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:27.465063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.549387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:27.553729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c361b5d-1cf5-456b-af65-c3d4c91b9ef0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.570306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353900e3-c857-4aec-8a7f-8ff09bbc76bf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.592172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.592896 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.593173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.593449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.594121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:27.594509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:27.595843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:27.606279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 204/407] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:27.606905 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.608521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.609294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.609721 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.610120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.611013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:27.611504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:27.613071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:27.622694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 204/408] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:27.623771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.664140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:27.723763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be76d6-bb69-4808-93b5-beb3d6dc5b68 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.727129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed6c7-372c-4d1c-a527-fb28fb4a2217 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.760357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.761047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.761329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.761604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1/groups/906fd3862ffb42ada255181e587e3119/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.762518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:27.763383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:27.763800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] RBAC: Authorizing `identity:create_grant(project_id=f33b9d002631429689e361f259c154d1, group_id=906fd3862ffb42ada255181e587e3119, role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:27.764040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:27.764480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:27.765267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:27.765951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:27.767542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:27.769641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 205/409] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:27.770748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.786057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 205/410] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 22 05:25:27 2022] PUT /identity/v3/projects/f33b9d002631429689e361f259c154d1/groups/906fd3862ffb42ada255181e587e3119/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:27.788368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:27.835217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:27.984987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc48077-f3b6-4db2-b872-90994c724886 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.989391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0432c58-9827-4542-a55f-9ffdaedd6c46 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:27.998055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:28.032029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359bb5e-4e86-45e0-af57-f9559d3e1bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:28.032997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359bb5e-4e86-45e0-af57-f9559d3e1bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.033482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359bb5e-4e86-45e0-af57-f9559d3e1bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.033931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359bb5e-4e86-45e0-af57-f9559d3e1bd6 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.035323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359bb5e-4e86-45e0-af57-f9559d3e1bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:28.035438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:28.036207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.036557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.036844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] PATH_INFO: `/v3/groups/c3176c73a4d34f60ac3b5db1cef4556e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.037501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359bb5e-4e86-45e0-af57-f9559d3e1bd6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:28.037973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] RBAC: Authorizing `identity:get_group(group_id=c3176c73a4d34f60ac3b5db1cef4556e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:28.039312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 206/411] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:28.039681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:28.040558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205ff34-3340-4d35-ba8d-53615542b689 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.040843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205ff34-3340-4d35-ba8d-53615542b689 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.041116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205ff34-3340-4d35-ba8d-53615542b689 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.041626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 206/412] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:27 2022] GET /identity/v3/groups/c3176c73a4d34f60ac3b5db1cef4556e => generated 229 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:28.042303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 207/413] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:28.047554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205ff34-3340-4d35-ba8d-53615542b689 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:28.047728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:28.151659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:28.161706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf9d73-09bf-41b5-bef8-e67b30c3d90f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:28.165668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205ff34-3340-4d35-ba8d-53615542b689 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:28.198210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da667a54-48c9-4da4-8094-486e52173c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:28.198646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da667a54-48c9-4da4-8094-486e52173c96 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.198927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da667a54-48c9-4da4-8094-486e52173c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.199202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da667a54-48c9-4da4-8094-486e52173c96 None admin] PATH_INFO: `/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.200462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da667a54-48c9-4da4-8094-486e52173c96 None admin] RBAC: Authorizing `identity:get_project(project_id=fb2b2de175ff44f7b8683cb1604be1f7)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:28.201924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da667a54-48c9-4da4-8094-486e52173c96 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:28.204015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 207/414] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:28 2022] GET /identity/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:28.204631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:28.205323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f714ab-dfaf-4121-9011-87cd9d9a18ad None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.205605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f714ab-dfaf-4121-9011-87cd9d9a18ad None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.205881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f714ab-dfaf-4121-9011-87cd9d9a18ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.206179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.206642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.207114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1/groups/906fd3862ffb42ada255181e587e3119/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.209712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f33b9d002631429689e361f259c154d1, group_id=906fd3862ffb42ada255181e587e3119, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:28.211525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:28.225882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b5f714ab-dfaf-4121-9011-87cd9d9a18ad None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:28.227257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b5f714ab-dfaf-4121-9011-87cd9d9a18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:28.228488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 208/415] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 22 05:25:28 2022] PUT /identity/v3/projects/f33b9d002631429689e361f259c154d1/groups/906fd3862ffb42ada255181e587e3119/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:28.229689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:28.273152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:28.321169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 208/416] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:28.322567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dda765-312a-4d2f-988f-1a95f0bc8bdb None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.322863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dda765-312a-4d2f-988f-1a95f0bc8bdb None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.323108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dda765-312a-4d2f-988f-1a95f0bc8bdb None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.324735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 209/417] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:28.330448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bf749-e2cc-4797-be4b-9033386b106a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.330727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bf749-e2cc-4797-be4b-9033386b106a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.330999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bf749-e2cc-4797-be4b-9033386b106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.364663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c41bf749-e2cc-4797-be4b-9033386b106a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:28.365891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c41bf749-e2cc-4797-be4b-9033386b106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:28.395943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4f20c-a612-4779-8971-dc2be136e4c7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:28.433120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:28.433723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.434002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.434275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] PATH_INFO: `/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7/groups/c3176c73a4d34f60ac3b5db1cef4556e/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.436199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb2b2de175ff44f7b8683cb1604be1f7, group_id=c3176c73a4d34f60ac3b5db1cef4556e, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:28.437644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:28.456525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 209/418] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 22 05:25:28 2022] PUT /identity/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7/groups/c3176c73a4d34f60ac3b5db1cef4556e/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:28.475345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 210/419] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:28.619279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:28.631359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:28.737509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6959649-1f66-4987-8458-255b6632c365 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:28.767464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bf749-e2cc-4797-be4b-9033386b106a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:28.777424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:28.778372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.778847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.779326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.790472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:28.796183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:28.797066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:28.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:28.800166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 210/420] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:28.806437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:28.897328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bf749-e2cc-4797-be4b-9033386b106a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:28.899551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:28.920068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:28.950674 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:28.952499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:28.954003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:28.954580 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:28.975999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:28.980024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386240b8-a426-4265-8326-8b1e1010a20c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:28.984515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:28.985477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:28.986270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:28.990313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 211/421] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:28.998736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:29.049960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:29.052055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:29.053202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:29.054363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:29.055454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:29.056213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:29.059793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:29.076622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 211/422] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:29.083993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:29.196480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505a87c-a602-4b34-b973-4a0cb405ba2b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:29.257352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add81f82-6988-4b20-bd06-e3bffaa7c2b8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:29.259503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:29.261253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:29.262602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:29.262884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:29.263900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:29.264215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:29.265646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:29.285357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 212/423] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:29.287912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:29.314781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:29.324247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:29.324881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:29.325175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:29.325454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:29.332136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:29.431725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:29.433208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:29.471414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffc24-934e-4302-a184-8dfe546295bb None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:29.524195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:29.524842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:29.525006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:29.525282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:29.620354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:29.622079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:29.650675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:29.656287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 212/424] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 05:25:29.662009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:29.699917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:29.792476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 213/425] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 05:25:29.799303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:29.817752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ed184-ace1-4956-9e8e-88ad578079a2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:29.855904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:29.856604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:29.856844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:29.857118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:29.858416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:29.860258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:29.863180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 213/426] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:29 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:29.868732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:29.991503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:29.996645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8614f7-44ed-4471-8a67-26202ceba7ce None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:30.016060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ecb50-d589-4a21-9e99-4964de7c4fab None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:30.039672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee963a1e-6390-4819-8b88-4747a2f94bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:30.040332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee963a1e-6390-4819-8b88-4747a2f94bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.040609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee963a1e-6390-4819-8b88-4747a2f94bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.040882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee963a1e-6390-4819-8b88-4747a2f94bc1 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.042033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee963a1e-6390-4819-8b88-4747a2f94bc1 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:30.043563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee963a1e-6390-4819-8b88-4747a2f94bc1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:30.045480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 214/427] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:29 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:30.047030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898847ab-9a8c-487c-a0c6-1485c95d65c2 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.047030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898847ab-9a8c-487c-a0c6-1485c95d65c2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.047256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898847ab-9a8c-487c-a0c6-1485c95d65c2 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.051707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 215/428] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:30.051707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898847ab-9a8c-487c-a0c6-1485c95d65c2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:30.066940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468ba5e-4376-4720-a4e5-46aadd0d2b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:30.067889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468ba5e-4376-4720-a4e5-46aadd0d2b87 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.068313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468ba5e-4376-4720-a4e5-46aadd0d2b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.068722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468ba5e-4376-4720-a4e5-46aadd0d2b87 None admin] PATH_INFO: `/v3/projects/5331948968af49e2bac622c653dc3e25` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.070029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468ba5e-4376-4720-a4e5-46aadd0d2b87 None admin] RBAC: Authorizing `identity:get_project(project_id=5331948968af49e2bac622c653dc3e25)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:30.071871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468ba5e-4376-4720-a4e5-46aadd0d2b87 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:30.074237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:30.074751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 214/429] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:29 2022] GET /identity/v3/projects/5331948968af49e2bac622c653dc3e25 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:30.077917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3408d-6564-4996-9029-70f7ee8e0018 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.078414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3408d-6564-4996-9029-70f7ee8e0018 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.078950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3408d-6564-4996-9029-70f7ee8e0018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.101625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-19c3408d-6564-4996-9029-70f7ee8e0018 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:30.103896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-19c3408d-6564-4996-9029-70f7ee8e0018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:30.175355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898847ab-9a8c-487c-a0c6-1485c95d65c2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:30.193236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 215/430] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:30.194591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3408d-6564-4996-9029-70f7ee8e0018 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:30.249810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:30.250888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.253189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.253675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.254970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=5331948968af49e2bac622c653dc3e25) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:30.256202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:30.258225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:30.274124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 216/431] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:30 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:30.361664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3408d-6564-4996-9029-70f7ee8e0018 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:30.397512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:30.412557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:30.413519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.413973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.414597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.416063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:30.417757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:30.419997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 216/432] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:30 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:30.428733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:30.461330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:30.511788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:30.549667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ec5de-ca5f-49fd-94c6-5139933de154 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:30.604637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8edb2c-b793-4cb9-9aea-bbd2247fb483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:30.605968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8edb2c-b793-4cb9-9aea-bbd2247fb483 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.606516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8edb2c-b793-4cb9-9aea-bbd2247fb483 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.606981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8edb2c-b793-4cb9-9aea-bbd2247fb483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.608358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8edb2c-b793-4cb9-9aea-bbd2247fb483 None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=5331948968af49e2bac622c653dc3e25) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:30.608850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8edb2c-b793-4cb9-9aea-bbd2247fb483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:30.610600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8edb2c-b793-4cb9-9aea-bbd2247fb483 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:30.619501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 217/433] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:30 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=5331948968af49e2bac622c653dc3e25&role.id=69140820db3a4dfea9428bdde1203a39 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:30.648467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a4db0-2baa-4421-8c55-0150ba65b005 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.649635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a4db0-2baa-4421-8c55-0150ba65b005 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.650104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a4db0-2baa-4421-8c55-0150ba65b005 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.652294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 218/434] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:25:30.662425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53742498-5d63-45f4-8ee7-ab3fb0313abf None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.664126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53742498-5d63-45f4-8ee7-ab3fb0313abf None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.664660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53742498-5d63-45f4-8ee7-ab3fb0313abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.667052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f970e8a-a78a-467c-8f3c-7efd7ab42445 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:30.689697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-53742498-5d63-45f4-8ee7-ab3fb0313abf None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:30.692177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-53742498-5d63-45f4-8ee7-ab3fb0313abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:30.711307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:30.712012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.712291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.712558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.728666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:30.735171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:30.736188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:30.736987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 217/435] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:30.744498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:30.788967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 219/436] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:30.797060 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843b00f-06e1-4e53-951b-7fa465d791f6 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.797347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843b00f-06e1-4e53-951b-7fa465d791f6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.797624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843b00f-06e1-4e53-951b-7fa465d791f6 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.799256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 220/437] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:30.804741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cab7a9-7836-414e-a399-668bade0805e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.805298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cab7a9-7836-414e-a399-668bade0805e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.805455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cab7a9-7836-414e-a399-668bade0805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.806834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:30.826766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-00cab7a9-7836-414e-a399-668bade0805e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:30.829005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-00cab7a9-7836-414e-a399-668bade0805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:30.852172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:30.873288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358931b9-539a-41ce-abe8-67c5c98647bf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:30.918065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:30.918683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:30.918958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:30.919226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:30.919928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:30.920236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:30.921620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:30.924426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 221/438] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:30.932076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 218/439] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:30.937665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cab7a9-7836-414e-a399-668bade0805e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.044579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cab7a9-7836-414e-a399-668bade0805e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.060088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.084221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.085397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.085680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.085956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.115986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.180912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:31.182472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:31.188307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ece7f3-b43e-4c8f-aaaa-1659cde4269c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.203846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.237129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.237787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.238151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.238347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.249895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:31.255494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:31.256842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:31.257754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 219/440] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:31.258523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.300576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 222/441] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 05:25:31.301255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.358809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.368394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf26f5-0626-4b79-8fd9-b1eb55f9912c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.402052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.402686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.402967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.403098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b821e0-0df6-4fb7-bac7-112a6dd3c3c7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.403214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.413165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:31.419200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:31.420277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 220/442] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:31.421063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.436482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.437022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.437294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.437564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.438221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] RBAC: Adding query filter params (name=member) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:31.438515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:31.439911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:31.447874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 223/443] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:31.448643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.454367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.522220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df7b3b-b0de-4f9d-b60b-de1963a15195 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.554616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cc2e8-0d94-46c5-892f-43dc72e56532 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.555889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.556527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.556805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.557072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.558184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:31.559686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:31.561299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 221/444] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:31.562100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.575719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.594159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.594782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.595063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.595318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.596026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:31.596330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:31.598078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:31.606614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 224/445] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:31.607398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.665739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.669453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a314a-d017-4dec-97fc-62dd4b296955 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.704331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.704866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.705139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.705404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.706824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bc989-6bf4-48ca-a106-98c6db873442 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.741203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.741831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.742105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.742419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.743521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:31.744974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:31.746697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 225/446] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:31.747284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.775612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:31.777113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:31.795067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.848692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a2bf-1e47-40c7-a6e5-0bb22b01be22 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:31.868964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:31.873028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 222/447] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:31.874014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:31.884870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:31.885482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:31.885756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:31.886037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:31.956908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:31.958332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:31.977833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a4adb-db4f-47ed-a3b3-d0dc0aae3f42 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:32.002989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:32.012315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:32.012924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:32.013196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:32.013466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:32.014191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=f33b9d002631429689e361f259c154d1) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:32.014488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:32.015961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:32.021659 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 223/448] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:32.022415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:32.079044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 226/449] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:32.085095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:32.096978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:32.144062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd98206-215a-40ac-9794-f37a0e907cb2 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:32.185778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:32.186535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:32.186814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:32.187079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] PATH_INFO: `/v3/projects/f33b9d002631429689e361f259c154d1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:32.188542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] RBAC: Authorizing `identity:get_project(project_id=f33b9d002631429689e361f259c154d1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:32.190005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:32.192745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 224/450] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:32 2022] GET /identity/v3/projects/f33b9d002631429689e361f259c154d1 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:32.200080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e33e9e-397c-4a0e-9c40-4d8ced48a798 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:32.201913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:32.237465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:32.238118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:32.238397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:32.238649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] PATH_INFO: `/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:32.239797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=fb2b2de175ff44f7b8683cb1604be1f7)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:32.241240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:32.243226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 227/451] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:25:32 2022] GET /identity/v3/projects/fb2b2de175ff44f7b8683cb1604be1f7 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:32.248805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:32.264468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:32.317936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf72a19-a426-4b89-bbdb-46fc0fc1ad95 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:32.347120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564a19b-3872-4d99-9db2-b2beff570bf3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:32.347997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:32.354119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:32.354759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:32.355037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:32.355306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] PATH_INFO: `/v3/groups/906fd3862ffb42ada255181e587e3119` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:32.356458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] RBAC: Authorizing `identity:get_group(group_id=906fd3862ffb42ada255181e587e3119)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:32.357925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:32.359484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 225/452] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:32 2022] GET /identity/v3/groups/906fd3862ffb42ada255181e587e3119 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:32.365426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:32.380555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:32.381204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:32.381482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:32.381753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] PATH_INFO: `/v3/groups/c3176c73a4d34f60ac3b5db1cef4556e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:32.382910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] RBAC: Authorizing `identity:get_group(group_id=c3176c73a4d34f60ac3b5db1cef4556e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:32.384447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:32.386044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 228/453] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 22 05:25:32 2022] GET /identity/v3/groups/c3176c73a4d34f60ac3b5db1cef4556e => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:32.391805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:32.469486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330e197-7190-40c8-8cee-3f77b932ea64 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:32.473504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:32.491661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dc844-3903-4f97-bdf6-84606693f076 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:32.503119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c21b24-c02f-4dd2-b4c5-04c0d8590542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:32.503768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c21b24-c02f-4dd2-b4c5-04c0d8590542 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:32.504053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c21b24-c02f-4dd2-b4c5-04c0d8590542 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:32.504325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c21b24-c02f-4dd2-b4c5-04c0d8590542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:32.505043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c21b24-c02f-4dd2-b4c5-04c0d8590542 None admin] RBAC: Adding query filter params (group.id=906fd3862ffb42ada255181e587e3119, role.id=69140820db3a4dfea9428bdde1203a39, scope.project.id=f33b9d002631429689e361f259c154d1) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:32.505349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c21b24-c02f-4dd2-b4c5-04c0d8590542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:32.506795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c21b24-c02f-4dd2-b4c5-04c0d8590542 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:32.512836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 226/454] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:32 2022] GET /identity/v3/role_assignments?group.id=906fd3862ffb42ada255181e587e3119&scope.project.id=f33b9d002631429689e361f259c154d1&role.id=69140820db3a4dfea9428bdde1203a39 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:32.525801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c737a-0364-4128-915c-56330954d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:32.526431 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c737a-0364-4128-915c-56330954d874 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:32.526704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c737a-0364-4128-915c-56330954d874 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:32.526970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c737a-0364-4128-915c-56330954d874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:32.527755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c737a-0364-4128-915c-56330954d874 None admin] RBAC: Adding query filter params (group.id=c3176c73a4d34f60ac3b5db1cef4556e, role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=fb2b2de175ff44f7b8683cb1604be1f7) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:32.528062 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c737a-0364-4128-915c-56330954d874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:32.529559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c737a-0364-4128-915c-56330954d874 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:32.536337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 229/455] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 22 05:25:32 2022] GET /identity/v3/role_assignments?group.id=c3176c73a4d34f60ac3b5db1cef4556e&scope.project.id=fb2b2de175ff44f7b8683cb1604be1f7&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:35.107238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c5f2-fba3-4641-becb-b09817230f1b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:35.107783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c5f2-fba3-4641-becb-b09817230f1b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:35.107911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c5f2-fba3-4641-becb-b09817230f1b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:35.109513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 227/456] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:35.114772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab6ca1-c192-4803-bfae-f32a95c8cd2b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:35.115194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab6ca1-c192-4803-bfae-f32a95c8cd2b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:35.115372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab6ca1-c192-4803-bfae-f32a95c8cd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:35.144812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c8ab6ca1-c192-4803-bfae-f32a95c8cd2b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:35.146006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c8ab6ca1-c192-4803-bfae-f32a95c8cd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:35.259560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 230/457] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:35.341469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdeb8c-7970-48d1-9a0c-b4a943f42cef None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:35.341797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdeb8c-7970-48d1-9a0c-b4a943f42cef None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:35.342028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdeb8c-7970-48d1-9a0c-b4a943f42cef None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:35.343690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 228/458] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:35.354059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024361-ecab-4a87-a7d6-ea2dccdba1d4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:35.354660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024361-ecab-4a87-a7d6-ea2dccdba1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:35.355120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024361-ecab-4a87-a7d6-ea2dccdba1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:35.382577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9d024361-ecab-4a87-a7d6-ea2dccdba1d4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:35.384340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9d024361-ecab-4a87-a7d6-ea2dccdba1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:35.402211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986571-8ace-42c5-a4a9-7c1d8085107a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:35.402520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986571-8ace-42c5-a4a9-7c1d8085107a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:35.402763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986571-8ace-42c5-a4a9-7c1d8085107a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:35.404490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 229/459] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:35.411889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2dcba-8e41-4b39-b30a-8a0bed13daef None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:35.412206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2dcba-8e41-4b39-b30a-8a0bed13daef None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:35.412498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2dcba-8e41-4b39-b30a-8a0bed13daef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:35.451493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b5c2dcba-8e41-4b39-b30a-8a0bed13daef None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:35.453107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b5c2dcba-8e41-4b39-b30a-8a0bed13daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:35.483300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 231/460] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:35.568979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:35.582016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 230/461] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:35.673010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d024361-ecab-4a87-a7d6-ea2dccdba1d4 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:35.683278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:35.806243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d024361-ecab-4a87-a7d6-ea2dccdba1d4 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:35.853861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:35.854358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:35.854636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:35.854927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:35.870547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:35.872044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:35.877380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2dcba-8e41-4b39-b30a-8a0bed13daef None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:35.896346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:35.897337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:35.898270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:35.898270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:35.898270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:35.898270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 232/462] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:35.905565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:36.000399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:36.007130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2dcba-8e41-4b39-b30a-8a0bed13daef None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.050629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.051281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.051562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.052259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.055457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab240-4c55-4bdc-95b0-9e73a06aaa98 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.067914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.069544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:36.089324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:36.090295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:36.091127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:36.091127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:36.091127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:36.095617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 231/463] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:36.096525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:36.102272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.103171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.103667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.104231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.105605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:36.106345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.108919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.131869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 233/464] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:36.131869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:36.160461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:36.229158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470304dd-81a7-4810-bc85-a4f185717463 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.264005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43d736-3c05-444e-8ec4-d4bb313bd119 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.277843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.277843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.277843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.278223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.290986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:36.295885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.297653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:36.315948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:36.316907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:36.317764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:36.317764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:36.317764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:36.317764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 232/465] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:36.318408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:36.327689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.328774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.329242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.329608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.330283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:36.330588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.332018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.351655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 234/466] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:36.351655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:36.451717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:36.455458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f3ddc-c085-44e6-8bf8-76819ab47889 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.470370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238452-5d43-4fa4-a891-2d66a8389b7c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.497656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f5f3c-6144-4133-b358-4f9ec7b9a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.498282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f5f3c-6144-4133-b358-4f9ec7b9a470 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.498559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f5f3c-6144-4133-b358-4f9ec7b9a470 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.498957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f5f3c-6144-4133-b358-4f9ec7b9a470 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.499864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f5f3c-6144-4133-b358-4f9ec7b9a470 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.501211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f5f3c-6144-4133-b358-4f9ec7b9a470 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.504151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.504766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.505039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.505305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.505951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:36.506248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.507617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.521481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 235/467] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:36.521481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:36.531056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 233/468] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:36 2022] POST /identity/v3/users => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:36.532527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3571a-0e4e-4126-92fe-3ff20075a2d2 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.532934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3571a-0e4e-4126-92fe-3ff20075a2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.533457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3571a-0e4e-4126-92fe-3ff20075a2d2 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.539056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 234/469] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:36.539056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3571a-0e4e-4126-92fe-3ff20075a2d2 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:36.576717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:36.668216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3571a-0e4e-4126-92fe-3ff20075a2d2 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.669609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5375-e32c-4319-b315-b2bab66680ba None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:36.703825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415334d-8d93-409d-aba8-dadefc2da06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.704348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415334d-8d93-409d-aba8-dadefc2da06b None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.704628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415334d-8d93-409d-aba8-dadefc2da06b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.704897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415334d-8d93-409d-aba8-dadefc2da06b None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.705554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415334d-8d93-409d-aba8-dadefc2da06b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.707035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415334d-8d93-409d-aba8-dadefc2da06b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.709312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22322-f234-4b21-870f-0c1aaa17788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:36.710170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22322-f234-4b21-870f-0c1aaa17788e None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.710585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22322-f234-4b21-870f-0c1aaa17788e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.711019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22322-f234-4b21-870f-0c1aaa17788e None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.711935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22322-f234-4b21-870f-0c1aaa17788e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:36.713493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22322-f234-4b21-870f-0c1aaa17788e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:36.735874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 235/470] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:36 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:36.740542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e7fd7-dd16-4e91-a64e-01acf035444b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.740542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e7fd7-dd16-4e91-a64e-01acf035444b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.740542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e7fd7-dd16-4e91-a64e-01acf035444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.740818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 236/471] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:36 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:36.742409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee454e59-7bec-4f78-8593-56dac0ea5762 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.742536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee454e59-7bec-4f78-8593-56dac0ea5762 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.744513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee454e59-7bec-4f78-8593-56dac0ea5762 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.744513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 237/472] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:36.750368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417ba34-54cb-4123-aac3-63ceda9c172d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:36.750694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417ba34-54cb-4123-aac3-63ceda9c172d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:36.750921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417ba34-54cb-4123-aac3-63ceda9c172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:36.763549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-216e7fd7-dd16-4e91-a64e-01acf035444b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:36.764921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-216e7fd7-dd16-4e91-a64e-01acf035444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:36.770959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:36.776822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-b417ba34-54cb-4123-aac3-63ceda9c172d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:36.779619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b417ba34-54cb-4123-aac3-63ceda9c172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:36.871650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 236/473] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:36.878464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 238/474] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:37.151507 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e7fd7-dd16-4e91-a64e-01acf035444b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:37.155087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417ba34-54cb-4123-aac3-63ceda9c172d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:37.190382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:37.304271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417ba34-54cb-4123-aac3-63ceda9c172d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:37.321903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e7fd7-dd16-4e91-a64e-01acf035444b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:37.339758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:37.340415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:37.340687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:37.340954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:37.352139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:37.353588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:37.368901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:37.369837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:37.371000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:37.371000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:37.371000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:37.371000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 239/475] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:37.377235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:37.399666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:37.400289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:37.400569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:37.400835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:37.401578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:37.402908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:37.422617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.sql.core [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 05:25:37.422617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 05:25:37.422617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [parameters: {'id': '6192f2bbb459418fac6dfb99f8903e7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 05:25:37.422617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81585) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application conn.query(q) Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application result.read() Aug 22 05:25:37.442036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return ref.to_dict() Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 05:25:37.443018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application next(self.gen) Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application yield resource Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application next(self.gen) Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise self.value Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application session.commit() Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 05:25:37.444485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self.session.flush() Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application compat.raise_( Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 05:25:37.445621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 05:25:37.446678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application conn.query(q) Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application result.read() Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 05:25:37.447711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [parameters: {'id': '6192f2bbb459418fac6dfb99f8903e7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:37.448496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 05:25:37.449349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:37.450167 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 237/476] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 22 05:25:37 2022] POST /identity/v3/roles => generated 144 bytes in 293 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:37.463214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:37.470717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:37.537238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:37.583542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40f76-996e-4ee1-863c-bdcdcd1fa800 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:37.622741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:37.623805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:37.624096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:37.624371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:37.625084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:37.630219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:37.630219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:37.645768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 240/477] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:37.646998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:37.658234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821dfc-c3e3-4c8e-9f7b-7fbdbe4a22e0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:37.742702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:37.743831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:37.744320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:37.744775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:37.761025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847aed2-faad-4076-86ca-8f19c9fee676 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:37.768090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:37.773731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:37.774621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 05:25:37.775393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:37.785165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 238/478] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:25:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:37.801919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c579b-fc2d-488f-b82d-71892c8f4468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:37.802550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c579b-fc2d-488f-b82d-71892c8f4468 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:37.802829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c579b-fc2d-488f-b82d-71892c8f4468 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:37.803098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c579b-fc2d-488f-b82d-71892c8f4468 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:37.803799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c579b-fc2d-488f-b82d-71892c8f4468 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:37.804708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:37.805627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c579b-fc2d-488f-b82d-71892c8f4468 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:37.815971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:37.839135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 241/479] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:37 2022] POST /identity/v3/users => generated 249 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:37.956752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:38.034421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6fc2a-b237-4dc1-8cd9-a25e0046b371 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:38.071397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7e355-85e0-4906-a55e-a3c397923307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:38.072059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7e355-85e0-4906-a55e-a3c397923307 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:38.072339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7e355-85e0-4906-a55e-a3c397923307 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:38.072610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7e355-85e0-4906-a55e-a3c397923307 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:38.073295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7e355-85e0-4906-a55e-a3c397923307 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:38.073600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7e355-85e0-4906-a55e-a3c397923307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:38.075078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7e355-85e0-4906-a55e-a3c397923307 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:38.091717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 239/480] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:38.888773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32401c2f-6e08-4ccc-94fe-16ba3665be86 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:38.889295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32401c2f-6e08-4ccc-94fe-16ba3665be86 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:38.889295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32401c2f-6e08-4ccc-94fe-16ba3665be86 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:38.890912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 242/481] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:38.896298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99a8ec-b666-47f6-b70f-72a0f4fa8620 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:38.896746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99a8ec-b666-47f6-b70f-72a0f4fa8620 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:38.896923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99a8ec-b666-47f6-b70f-72a0f4fa8620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:38.917394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-ba99a8ec-b666-47f6-b70f-72a0f4fa8620 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:38.918600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ba99a8ec-b666-47f6-b70f-72a0f4fa8620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:38.988938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 240/482] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:39.087249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb010a7c-35ce-47d6-baf4-2644dce962c1 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.087617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb010a7c-35ce-47d6-baf4-2644dce962c1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.087834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb010a7c-35ce-47d6-baf4-2644dce962c1 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.089196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 243/483] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:39.103789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46374a-25d6-41e1-a60b-51ecc606a927 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.104076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46374a-25d6-41e1-a60b-51ecc606a927 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.104350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46374a-25d6-41e1-a60b-51ecc606a927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.129164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bc46374a-25d6-41e1-a60b-51ecc606a927 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:39.130403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bc46374a-25d6-41e1-a60b-51ecc606a927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:39.201211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:39.236098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 241/484] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:39.295528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880f837-1d4b-4a89-8b86-2362c1e8e8f5 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.295877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880f837-1d4b-4a89-8b86-2362c1e8e8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.296113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880f837-1d4b-4a89-8b86-2362c1e8e8f5 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.297720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 244/485] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:39.302863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14da72-a6a3-4cef-8f19-c66836e5e071 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.303154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14da72-a6a3-4cef-8f19-c66836e5e071 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.303432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14da72-a6a3-4cef-8f19-c66836e5e071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.332602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7c14da72-a6a3-4cef-8f19-c66836e5e071 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:39.333810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7c14da72-a6a3-4cef-8f19-c66836e5e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:39.411822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 242/486] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:39.419797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880f837-1d4b-4a89-8b86-2362c1e8e8f5 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:39.560393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880f837-1d4b-4a89-8b86-2362c1e8e8f5 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:39.595976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14da72-a6a3-4cef-8f19-c66836e5e071 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:39.632459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:39.633139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.633411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.633888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.658157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:39.672610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:39.673541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:39.674321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 245/487] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:39.681411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:39.734737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14da72-a6a3-4cef-8f19-c66836e5e071 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:39.751841 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:39.771593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:39.772223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.772500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.772769 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.783568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:39.788293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:39.789240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:39.790037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 243/488] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:39.796578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:39.816037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:39.884730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230fca2-8bec-48f6-bc02-1625babfab7c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:39.931456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4ce68-fa7a-4228-8ff2-1af933c1a77d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:39.938509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:39.939752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.940270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.940915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.942361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:39.942873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:39.945472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:39.956676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 246/489] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:39.958233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:39.971600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:39.972218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:39.972855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:39.973122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:39.973790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:39.974086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:39.975468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:39.985511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 244/490] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:39.990302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.032141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.101065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948042d0-7482-4bb4-8b60-30b09f5daa54 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.105320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb501a0-5c03-4212-b7b6-f8d8c2a8ef60 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.121488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.144110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.145319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.145624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.146043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.146264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.146542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.146813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.147092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.158585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:40.159384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.160893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:40.164846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:40.165732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:40.166520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:40.169144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 247/491] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:40.170128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:40.180467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:40.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:40.182213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:40.182213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:40.182213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:40.182213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 245/492] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:40.182981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.278410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce49f60-8a79-49a8-901d-204ea4579c9a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.279964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.291362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dca75-411a-4e03-bc5e-8b990e3832ba None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.317460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904becf-c5fe-4b94-8d9e-825af18435d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.318151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904becf-c5fe-4b94-8d9e-825af18435d7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.318368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904becf-c5fe-4b94-8d9e-825af18435d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.318633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904becf-c5fe-4b94-8d9e-825af18435d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.328727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.329354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.329633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.329905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.330243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904becf-c5fe-4b94-8d9e-825af18435d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.330577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:40.330880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.331760 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904becf-c5fe-4b94-8d9e-825af18435d7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.332316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.344958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 246/493] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:40.344958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-2904becf-c5fe-4b94-8d9e-825af18435d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:40.351149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:40.352093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:40.352914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:40.352914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:40.352914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:40.352914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 248/494] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:40.358733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e0a3c-cd11-4e34-913e-d5a8535e9faa None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.358733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e0a3c-cd11-4e34-913e-d5a8535e9faa None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.358733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e0a3c-cd11-4e34-913e-d5a8535e9faa None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.358733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 249/495] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:40.358733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e0a3c-cd11-4e34-913e-d5a8535e9faa None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.391219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.461268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da04140-1996-409b-ba5b-9914c429720c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.466286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e0a3c-cd11-4e34-913e-d5a8535e9faa None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.497926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.498534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.498806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.499073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.499796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:40.500076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.501172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c428ab7-f5c5-4df6-a774-c504f2af98ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.501504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.501995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c428ab7-f5c5-4df6-a774-c504f2af98ee None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.502360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c428ab7-f5c5-4df6-a774-c504f2af98ee None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.502733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c428ab7-f5c5-4df6-a774-c504f2af98ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.513501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c428ab7-f5c5-4df6-a774-c504f2af98ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.518466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 247/496] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:40.518601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c428ab7-f5c5-4df6-a774-c504f2af98ee None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.518693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.533184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-5c428ab7-f5c5-4df6-a774-c504f2af98ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:40.534011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:40.535059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:40.536131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:40.536131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:40.536131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:40.536131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 250/497] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:40.536397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61116-69a6-4975-b5bf-b6e85a05d509 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.536522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61116-69a6-4975-b5bf-b6e85a05d509 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.536793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61116-69a6-4975-b5bf-b6e85a05d509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.554206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a9d61116-69a6-4975-b5bf-b6e85a05d509 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:40.555550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a9d61116-69a6-4975-b5bf-b6e85a05d509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:40.619167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 251/498] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:40.620370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e87bc-5cd0-4676-8172-ae8f626c2fc0 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.620681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e87bc-5cd0-4676-8172-ae8f626c2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.620982 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e87bc-5cd0-4676-8172-ae8f626c2fc0 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.623725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 252/499] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:40.623725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e87bc-5cd0-4676-8172-ae8f626c2fc0 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.625830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41524-0c30-450d-9056-9fc28d7c44ea None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.641800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.662512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.663284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.663649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.664047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.665124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:40.665525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.667080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.683720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 248/500] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:40.683720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.745811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e87bc-5cd0-4676-8172-ae8f626c2fc0 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.752208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.783531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb639a01-d99a-47a7-8a3d-a68cc30c6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.784201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639a01-d99a-47a7-8a3d-a68cc30c6a84 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.784478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639a01-d99a-47a7-8a3d-a68cc30c6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.784745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639a01-d99a-47a7-8a3d-a68cc30c6a84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.785556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe5ce7-d23a-4ca5-a10a-6da47c02861c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.792103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb639a01-d99a-47a7-8a3d-a68cc30c6a84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.793562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb639a01-d99a-47a7-8a3d-a68cc30c6a84 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-fb639a01-d99a-47a7-8a3d-a68cc30c6a84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:40.806299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:40.807408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:25:40.808535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:40.812130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 253/501] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:40.813375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7260dd-a763-4170-91ae-c8d638fa3155 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.813658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7260dd-a763-4170-91ae-c8d638fa3155 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.813932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7260dd-a763-4170-91ae-c8d638fa3155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.823755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.824491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.824857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.825219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:40.825987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:40.826374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:40.827885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:40.832044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3d7260dd-a763-4170-91ae-c8d638fa3155 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:40.833241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3d7260dd-a763-4170-91ae-c8d638fa3155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:40.840669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 249/502] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:40.845835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.902322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.908423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 254/503] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:40.909331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7260dd-a763-4170-91ae-c8d638fa3155 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:40.962087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8309a6-e32d-4b93-850a-579dca66a7a9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:40.992655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:40.998107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:40.998911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:40.999280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:40.999661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.006912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.008774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:41.021252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:41.022373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 05:25:41.023456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:41.027130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 250/504] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:41.028027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.029571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7260dd-a763-4170-91ae-c8d638fa3155 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.069052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.069665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.070146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.070283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.070943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:41.071243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.072709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.116355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 255/505] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:25:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:41.116355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.135495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a13c48-0cb5-417c-a204-8a7635448d3d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.169117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.169753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.170034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.170303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.173085 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:41.178043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.179530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:41.190521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:41.191568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:41.192464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 251/506] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:41.193015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.218969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf256-e311-41a8-bcd2-c642478c17cf None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.245764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:41.252799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.253428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.253719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.253987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.264375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.265869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:41.280208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:41.281282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:41.282127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:41.282127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:41.282127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:41.282127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 256/507] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:41.282127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.295079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e69a0a-420f-47d4-8a1b-432a1aac4e21 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.328130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.328740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.329017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.329289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.330028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:41.330333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.331810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.367744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 252/508] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:41.368625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.385199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23d25-2377-4707-908e-6192e5624dc8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.394522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:41.427047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.427564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.427868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.428139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.475618 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568fe25-6294-42d3-bbfb-8929b0a62d5e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.486168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:41.497615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.499094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.509987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.510614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.510889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.511163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.520921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:41.525626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:41.526646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 253/509] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:41.527541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.601070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 257/510] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:41.602016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.613424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:41.629930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a9cf0-0e7d-436c-82bc-7719570b0b94 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.662513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.663118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.663390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.663695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.664838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:41.665174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.666647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.684490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 254/511] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:41.685236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.708444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a113880-dc11-4e9b-8d42-2911628bc671 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.718730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:41.743824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.744434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.744707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.744968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.745611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:41.745902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.747268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.759207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 258/512] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:41.760023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.796579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707ace9-ed41-4077-992f-bd5762cc2465 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.823293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:41.830612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.831234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.831513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.831814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.858377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215b6c1-66fc-486d-a68a-a3fda66b2c8b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:41.893929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:41.894547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:41.894822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:41.895087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:41.895880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:41.896182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.897600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.905857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:41.905990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 259/513] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:41.906804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:41.907325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:41.937454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.010530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3130c-bac6-47d9-9642-b6f7e57fec0d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.017278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 255/514] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:42.018116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.048553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.056006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.056657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.056935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.057202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.067680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.069155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:42.083752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:42.084660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:42.085496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:42.085496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.085496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:42.085496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 260/515] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:42.085733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.121962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659e716-7a36-43f9-978e-18570f196cc6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.148988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.157583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.158190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.158462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.158729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.187963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f1e56-af37-4f9d-8703-bdcc09a792fd None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.220578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.221186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.221468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.221734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.222732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.224235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.226956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.228455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.255794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 261/516] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:25:42 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:25:42.255794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.292227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.355506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 256/517] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:42.356414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.371198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cff0b-5e9a-4162-b1d7-57cd637d016a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.396547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.416107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.416757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.417034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.417308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.431267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.432780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:42.449127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:42.450040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:42.450876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:42.450876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.450876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:42.450876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 262/518] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:42.451263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.482606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ff1c-a688-433f-a310-dc3f6d5cb18c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.520119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.520747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.521130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.521399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.532076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.533757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.542480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:42.550291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:42.551181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:42.552405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:42.552405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.552405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:42.552405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 257/519] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:42.552405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.558832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa7a2c-fbc6-499a-80be-47f96a5b9c91 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.593572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.594186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.594465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.594735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.595386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:42.595754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.597133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.610383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 263/520] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:42.611197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.641729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.665510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7379-bf83-495e-89cb-7f62f4081385 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.715410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.716218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.716505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.716831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.719873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88cfa-fca6-40c8-95bc-4b27e9361174 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.731322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.733182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:42.748114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:42.749029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:42.749860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:42.749860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:42.749860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:42.749860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 258/521] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:42.750706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.765058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.765717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.765977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.766181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.766928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:42.767393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.768936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.781463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 264/522] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:42.781463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.801570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.860740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b15bb9-e564-4359-a4c7-e56643d150cd None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.892235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:42.897629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.898551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.898981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.899348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.900290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:42.900632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.902231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.909428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47905c-9fa0-48ee-8931-b89463d55e72 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:42.915778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 259/523] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:42.916606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:42.951475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:42.952419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:42.952698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:42.952970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:42.959727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:42.961247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:42.973602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:42.974596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 265/524] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:42.976002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.027239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314a0e1-f14c-421a-8b3a-6a0093528fa5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.049594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.061582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.062193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.062467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.062731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.063382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:43.063720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.065111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.082304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 260/525] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:43.082304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.087942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11199f08-803d-464c-ac40-f05ba5a61062 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.123112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.123799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.124103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.124370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.132458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.133992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.136142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:43.145842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:43.146759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 266/526] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:43.147754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.191625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcaf51-67a1-4ab9-b452-fe49cc9481dc None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.224742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.225292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.225574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.225840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.232658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.234129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:43.246082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:43.247142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:43.248234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:43.250871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcde9-6b93-4976-8748-9ca16985980a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.252059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 261/527] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:43.252929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.280812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.287598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.288365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.288644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.289749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.290462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:43.290767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.292228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.311474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 267/528] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:43.312160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.368450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20370f5-7a42-4080-a37f-e9acc4d370f4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.383445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.408696 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.409695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.409969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.410238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.417610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.417723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9485578-a403-466b-a8a0-0138e5cadc21 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.419081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:43.431616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:43.432461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:43.433288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 262/529] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:43.433736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.456025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.456643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.456922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.457468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.457913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:43.458224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.459697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.484550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 268/530] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:43.484550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.533351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d653085-aa41-49de-bffa-ac8456548db0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.542172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.568209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.568820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.569097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.569761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.570481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:43.570787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.573007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.590443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 263/531] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:43.591250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.600247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef52a5-8a1f-4d4e-b8e2-c5c740ccaea1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.624859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.636327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.637018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.637360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.637686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.638451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=a3b919bcbb104fa3bf11847b6a35a221) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:43.638752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.640237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.655610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 269/532] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/role_assignments?user.id=a3b919bcbb104fa3bf11847b6a35a221&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:43.655610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.710390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccac3f-bd61-49a3-9973-6c8d66ecb00a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.752368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.753003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.753342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.753673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.754449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:43.754814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.756395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.769539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 264/533] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:43.770470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:43.783704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251e57-ce03-45e5-8305-4e3840779df6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.797855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.820772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.821414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.821679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.821953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.882051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:43.893142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa87756-676c-4a21-ba83-3b5d28bd526c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:43.908360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.910278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.938663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:43.939332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:43.939725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:43.939999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:43.941069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=69d9ee15b5fa43248fbdca69c3c91a6b) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:43.941378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:43.942930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:43.949761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 265/534] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/role_assignments?user.id=69d9ee15b5fa43248fbdca69c3c91a6b&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:43.950596 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:44.063133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65f78-6f09-40f0-8a8c-f5cd1e22e8ee None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:44.081123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 270/535] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:44.088155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:44.114229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:44.142253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856d15-4d90-4fa5-8348-442f95b4bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:44.142889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856d15-4d90-4fa5-8348-442f95b4bd7a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:44.143164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856d15-4d90-4fa5-8348-442f95b4bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:44.143433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856d15-4d90-4fa5-8348-442f95b4bd7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:44.144207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856d15-4d90-4fa5-8348-442f95b4bd7a None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=3fcf79bef8d542d9adbfce89931032c0) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:44.144516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856d15-4d90-4fa5-8348-442f95b4bd7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:44.146041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856d15-4d90-4fa5-8348-442f95b4bd7a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:44.153200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 266/536] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:43 2022] GET /identity/v3/role_assignments?user.id=3fcf79bef8d542d9adbfce89931032c0&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:44.223365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49b08f-0376-4c3f-b52f-59c53d1e7787 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:44.223719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49b08f-0376-4c3f-b52f-59c53d1e7787 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:44.223936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49b08f-0376-4c3f-b52f-59c53d1e7787 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:44.225440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 267/537] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:44.231599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c9ece-67df-4801-b79d-7dd9fdac0e20 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:44.231921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c9ece-67df-4801-b79d-7dd9fdac0e20 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:44.232128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c9ece-67df-4801-b79d-7dd9fdac0e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:44.257302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c86c9ece-67df-4801-b79d-7dd9fdac0e20 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:44.258576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c86c9ece-67df-4801-b79d-7dd9fdac0e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:44.267250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd7105-abb7-4d08-87f9-ee3cb37e0329 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:44.284025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:44.307739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:44.308578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:44.308957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:44.309327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:44.321543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:44.323044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:44.338455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:44.339605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:44.340633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:44.340633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:44.340633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:44.343598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 271/538] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:44.350740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:44.376778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 268/539] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:44.424469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:44.463994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3b6a-3e92-4cae-8052-1d3c4d0823d8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:44.500400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:44.501105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:44.501444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:44.501773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:44.502483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:44.502846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:44.504388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:44.520919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 272/540] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:44.522370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c9ece-67df-4801-b79d-7dd9fdac0e20 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:44.639672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c9ece-67df-4801-b79d-7dd9fdac0e20 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:44.682040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:44.682680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:44.682958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:44.683225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:44.690111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:44.691587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:44.702714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:44.703770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:44.704802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 269/541] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:44.710577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:44.751863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:44.767550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:44.913805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd08ff7-d070-4eaf-84fc-d5395669ec50 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:44.952066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:44.952685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:44.952964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:44.953236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:44.953934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:44.954237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:44.955691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:44.959925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:44.961513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5c5f5-cceb-4c26-aff3-bddd9345f6a2 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:44.985659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 273/542] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:45.001927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:45.011565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:45.015471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:45.015881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:45.016242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:45.035055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:45.041111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:45.042186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:45.043187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:45.049822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 270/543] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:45.054270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:45.126314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:45.156753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7bfb2-ef88-41e4-8239-b2ea71dd1a7d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:45.191283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:45.191922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:45.192249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:45.192517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:45.193184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:45.193481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:45.194861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:45.212374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 271/544] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:45.212374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:45.259646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae2780-d4ed-48a0-9656-f1c63d450204 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:45.323153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c7d53-5f39-48be-92f9-8be6d34941c9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:45.332829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:45.333648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:45.334063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:45.334493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:45.335368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=0a5c594aafc14c1e86ffa9702f9be964) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:45.335887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:45.337453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:45.349596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 274/545] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:45 2022] GET /identity/v3/role_assignments?user.id=0a5c594aafc14c1e86ffa9702f9be964&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:45.358097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:45.363651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:45.364381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:45.364755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:45.365036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:45.377632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:45.379165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:45.395886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:45.396852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:45.397673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:45.397673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:45.397673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:45.401573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 272/546] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:45.404085 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:45.466265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:45.536845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182ecec-fe3a-4104-85c9-2e9c74cf3dce None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:45.589857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:45.590495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:45.590776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:45.591049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:45.591741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:45.592049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:45.593453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:45.607563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 275/547] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:45.613095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:45.728203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1f26a-d37a-4039-a944-3d54ea21005c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:45.762670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:45.763307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:45.763601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:45.763881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:45.771602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:45.773059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:45.784041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:45.784930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:25:45.786384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:45.789924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 273/548] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:45.791322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:45.916426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804fd1f-a538-457d-b5ba-968c101b3451 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:45.931626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:45.963478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df956c5-4e08-410d-b31b-7295b8c0cdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:45.964127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df956c5-4e08-410d-b31b-7295b8c0cdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:45.964403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df956c5-4e08-410d-b31b-7295b8c0cdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:45.964668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df956c5-4e08-410d-b31b-7295b8c0cdb3 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:45.965404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df956c5-4e08-410d-b31b-7295b8c0cdb3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:45.965703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df956c5-4e08-410d-b31b-7295b8c0cdb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:45.967155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df956c5-4e08-410d-b31b-7295b8c0cdb3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:45.993519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 276/549] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 22 05:25:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:45.999133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:46.075630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:46.134927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09824-5647-467c-bfb8-fcef2a06555f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:46.169120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:46.169964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.170409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.170813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.211096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d4f8-7cf8-4183-9672-aca1dae86a26 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.211306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d4f8-7cf8-4183-9672-aca1dae86a26 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.211420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d4f8-7cf8-4183-9672-aca1dae86a26 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.213322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 277/550] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:46.217957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0b221-30aa-42a6-b2b0-12c5401d8c5d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.218234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0b221-30aa-42a6-b2b0-12c5401d8c5d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.218521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0b221-30aa-42a6-b2b0-12c5401d8c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.241140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:46.242930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:46.263526 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0fa0b221-30aa-42a6-b2b0-12c5401d8c5d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:46.264722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0fa0b221-30aa-42a6-b2b0-12c5401d8c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:46.296471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:46.355518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 274/551] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:46.357505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:46.426121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 278/552] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:46.427590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a56d68-6988-4eaf-88f3-577aff121d67 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.427895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a56d68-6988-4eaf-88f3-577aff121d67 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.428164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a56d68-6988-4eaf-88f3-577aff121d67 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.429700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 279/553] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:46.441058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393abe74-d3c6-467a-b4c6-d008cacb3efb None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.441337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393abe74-d3c6-467a-b4c6-d008cacb3efb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.441610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393abe74-d3c6-467a-b4c6-d008cacb3efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.475430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-393abe74-d3c6-467a-b4c6-d008cacb3efb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:46.479461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-393abe74-d3c6-467a-b4c6-d008cacb3efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:46.510570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434441b-5312-4e1d-ad50-8320b7a1cce9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:46.561950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:46.562513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.562791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.563062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.563293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 280/554] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:46.564566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b53ee-3d37-42bc-bd2f-ef799744f9fe None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.564846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b53ee-3d37-42bc-bd2f-ef799744f9fe None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.565116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b53ee-3d37-42bc-bd2f-ef799744f9fe None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.572016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 281/555] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:46.577394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6bd39-cdb7-4f58-be79-f429e4bb6400 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.577677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6bd39-cdb7-4f58-be79-f429e4bb6400 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.578008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6bd39-cdb7-4f58-be79-f429e4bb6400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.578351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:46.579892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:46.582716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:46.598608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:46.599619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:46.601223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:46.601223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:46.601223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:46.601223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 275/556] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:46.607449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:46.612554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-fbb6bd39-cdb7-4f58-be79-f429e4bb6400 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:46.613739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fbb6bd39-cdb7-4f58-be79-f429e4bb6400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:46.664951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:46.729987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ba340-3df0-4720-8260-1b106e78a9e5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:46.764515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 282/557] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:46.765806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6bd39-cdb7-4f58-be79-f429e4bb6400 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:46.776530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:46.777475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.777586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.777792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.778458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:46.778765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:46.780327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:46.806528 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 276/558] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:46.816609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:46.879768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6bd39-cdb7-4f58-be79-f429e4bb6400 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:46.917784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:46.918702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:46.919297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:46.919799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:46.930696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:46.935868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:46.936777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:46.937570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:46.940377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:46.946567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 283/559] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:46.947935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:46.968468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:46.989112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1129e-163a-4f42-9f41-ba246a1ced0b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.024854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.025493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.025770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.026036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.034896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.036566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:47.049832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:47.051200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:47.052557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:47.056129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 277/560] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:47.057032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.063086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f1b5-05e5-4e09-a0de-0fe4144b93d9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.099690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.100790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.101220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.101523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.113292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:47.118493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:47.119446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:47.120386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 284/561] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:47.121103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.176926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065985c5-6dec-48a2-b1a2-2b5b40b6e890 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.194623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.212843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.214665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.215377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.215768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.216010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.216604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:47.216908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.218285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.227212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 278/562] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:47.227838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.244093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b67e6f-9e8b-4014-a6e9-4e646694df56 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.279841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.280490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.280889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.281165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.282241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:47.282725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.284175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.297741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 285/563] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:47.299030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.343225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4dfc-8ba9-4f18-986d-3f73bf8b4d55 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.382152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.382845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.383373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.383495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.395074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:47.401053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:47.402001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 279/564] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:47.404544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.410111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff193fd-bc14-44db-b063-4e81d68c658b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.445311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.447608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.448281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.448623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.448899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.449577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:47.449960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.451252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.453885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.461672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 286/565] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:47.462526 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.515519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea97a5-80ce-4060-8461-0b60c1e938cf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.548400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.549031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.549312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.549584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.560070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.561570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.563157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25971640-1223-4104-a4d3-9410823a9f33 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:47.575020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:47.576180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:47.577220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:47.577220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:47.577220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:47.577220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 280/566] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:47.577220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.600289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.600894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.601174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.601501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.602229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=b63e2db3e07c4471be6928ff31334816) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:47.602540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.604016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.609316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 287/567] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/role_assignments?user.id=b63e2db3e07c4471be6928ff31334816&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:47.610076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.674291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.677176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.685261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e55da-9ef7-4420-9496-c920dba17233 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.719712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365c2ea-a33f-4b90-b5a3-1e4b27df7975 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.721477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b887c-08bc-454d-9791-a8fbafcd9332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.722180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b887c-08bc-454d-9791-a8fbafcd9332 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.723529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b887c-08bc-454d-9791-a8fbafcd9332 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.723529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b887c-08bc-454d-9791-a8fbafcd9332 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.723971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b887c-08bc-454d-9791-a8fbafcd9332 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:47.724338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b887c-08bc-454d-9791-a8fbafcd9332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.725837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b887c-08bc-454d-9791-a8fbafcd9332 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.739633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 281/568] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:47.739633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9060-764c-4406-8360-af8db7e92bfd None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.739633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9060-764c-4406-8360-af8db7e92bfd None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.739633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9060-764c-4406-8360-af8db7e92bfd None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.739633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 282/569] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:47.739633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c9060-764c-4406-8360-af8db7e92bfd None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.778531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.779232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.779515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.779866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.791456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.792940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:47.807399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:47.808368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:47.809248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:47.809248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:47.809248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:47.809248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 288/570] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:47.809464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:47.854581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c9060-764c-4406-8360-af8db7e92bfd None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.889593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284e2c6-0b24-4b33-9d7a-43346c21895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.890218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284e2c6-0b24-4b33-9d7a-43346c21895e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.890499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284e2c6-0b24-4b33-9d7a-43346c21895e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.890772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284e2c6-0b24-4b33-9d7a-43346c21895e None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.891424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284e2c6-0b24-4b33-9d7a-43346c21895e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:47.891760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284e2c6-0b24-4b33-9d7a-43346c21895e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.893271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284e2c6-0b24-4b33-9d7a-43346c21895e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.909445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 283/571] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:47.910823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785ba1c-85e9-4fed-b4f3-f86f8d1a47d2 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.911174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785ba1c-85e9-4fed-b4f3-f86f8d1a47d2 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.911522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785ba1c-85e9-4fed-b4f3-f86f8d1a47d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.924658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5c1d-f64b-4486-ab8f-0532b299b039 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:47.931069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2785ba1c-85e9-4fed-b4f3-f86f8d1a47d2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:47.932391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2785ba1c-85e9-4fed-b4f3-f86f8d1a47d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:47.940198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.940520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:47.963196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:47.963867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:47.964146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:47.964480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:47.976659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:47.978230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:47.992503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:47.993493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:47.994366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:47.994366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:47.994366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:47.994366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 289/572] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:47.994615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.015964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 284/573] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:48.015964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785ba1c-85e9-4fed-b4f3-f86f8d1a47d2 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.110753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8fc7c-5ed4-4caf-8164-d3df303d48f5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.145397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.146069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.146346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.146615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.147019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785ba1c-85e9-4fed-b4f3-f86f8d1a47d2 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.147309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:48.147646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.149129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.167268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 290/574] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:48.167268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.196886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:48.197111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.198643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.199067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.199529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.208892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.210604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:48.224000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:48.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:25:48.225993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:48.228380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 285/575] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:48.229368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.232619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:48.283220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f0374-f2e2-433c-ba37-19a690f55ec0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.322158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.322920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.323291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.323687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.324430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:48.324833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.326297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.339698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 291/576] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:48.339852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5714bf-67a0-4945-b3aa-91d0721193b0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.339937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.376038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.376675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.376956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.377231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.384822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.386284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:48.397138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:48.398046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 286/577] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:48.399051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.437175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:48.454253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228861e-694e-4cf3-805f-39fab97de24f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.465865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:48.489398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.490099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.490438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.490757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.491551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:48.492014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.493444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.508812 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449dbcf-1c12-4ed2-b9d9-56f1bbcda444 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.515509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 292/578] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:48.515509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.549617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.550399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.550797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.551178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.562474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:48.567862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:48.568933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:48.569920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:48.572316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 287/579] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:48.573235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.629792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45886b6d-05e4-4912-8703-851f1b5bd251 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.665822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.666592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.666935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.667213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.675876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.677361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.686350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62714493-5dcd-4665-b428-efbc2b9b56ff None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:48.690517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:48.691614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:25:48.692723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:48.697230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 293/580] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:48.697954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.702361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:48.714803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:48.729423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.730082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.730364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.730641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.731647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:48.731757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.733311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.757148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 288/581] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:48.757148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.813551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd431-0532-4afd-b48d-014c26b37761 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.849126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.849762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.850042 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.850314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.857251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2864d6-6516-43d4-a0c8-464f9fe13c4d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:48.861151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.862618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:48.877479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:48.878617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:48.879886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:48.879886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:48.879886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:48.883335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 294/582] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:48.884231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.894205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:48.897406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:48.897406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:48.897406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:48.897406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:48.897406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:48.897826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:48.911554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 289/583] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:48.912617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:48.959314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:48.963130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.010549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3aa3a2-d31e-4c99-9774-f386b7e178a1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.022017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d2ca0-0769-4234-8875-4e8a3b795422 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.045829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.046448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.046730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.047003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.047772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:49.048078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.049508 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.061279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.062084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.062461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.062833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.071960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 295/584] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:49.073473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.075198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.076783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:49.091481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:49.092699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:49.093577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:49.093577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:49.093577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:49.093577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 290/585] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:49.093972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.184785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1bb34-8f50-4c6a-b3fa-7c5f7c42f637 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.190320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.198813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e2e98-0c50-4900-b737-192171ab499f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.201336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.221677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.222658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.222938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.223204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.223879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:49.224180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.225542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.237267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 296/586] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:49.238070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.245548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.246429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.246884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.247327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.259710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.261602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:49.278005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:49.278985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:49.284779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:49.284779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:49.284779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:49.284779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 291/587] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:49.284779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.350890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea35294-be06-4419-adbd-ecfd714afd5e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.386606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.387247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.387448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b7c5f-a200-40dc-a13c-662841b43283 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.387551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.387812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.399400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.400877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:49.415160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:49.416161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:49.417079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:49.417079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:49.417079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:49.417079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 297/588] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:49.417079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.428690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.429313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.430097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.430383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.431055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:49.431700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.433104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.450960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 292/589] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:49.450960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.452023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.457962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.529542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db8d0-90c2-41b8-8df4-129d54c9577c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.548723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124447c-38c3-4696-a3b0-d536498484af None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.561072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.561707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.561979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.562245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.568972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.570466 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:49.581955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:49.582972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:49.583906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:49.584817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.584817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.584988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 298/590] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:49.585056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.585056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.585194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.585272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:49.585272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.585939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.597787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 293/591] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:49.598425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.664883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.669329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.684379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d653a4-ca1a-4ea0-ba0d-c9fe0c2a4139 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.699430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a137e12-db13-44cc-af85-34b355b022ae None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.718002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee329f7-03ae-473e-8057-4c17523c97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.718623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee329f7-03ae-473e-8057-4c17523c97c4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.718909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee329f7-03ae-473e-8057-4c17523c97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.719184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee329f7-03ae-473e-8057-4c17523c97c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.719930 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee329f7-03ae-473e-8057-4c17523c97c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:49.720235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee329f7-03ae-473e-8057-4c17523c97c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.721628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee329f7-03ae-473e-8057-4c17523c97c4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.732483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.733115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.733394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.733678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.734045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 299/592] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:49.735215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9e250-066f-4943-a784-9fc46745fbf7 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.735536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9e250-066f-4943-a784-9fc46745fbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.735845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9e250-066f-4943-a784-9fc46745fbf7 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.737002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 300/593] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:49.737683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9e250-066f-4943-a784-9fc46745fbf7 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.742017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.743538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:49.754645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:49.755549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 294/594] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:49.756469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.845068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9e250-066f-4943-a784-9fc46745fbf7 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.857891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31db04-decd-4b27-950c-faa6d54efcdf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:49.878696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.879305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.879605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.879883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.880574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:49.880882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.882324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.884253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.893061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9feda6-00d1-4a13-8982-e90067c769ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:49.893692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9feda6-00d1-4a13-8982-e90067c769ea None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.893788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 301/595] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:49.893975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9feda6-00d1-4a13-8982-e90067c769ea None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.894254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9feda6-00d1-4a13-8982-e90067c769ea None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.896294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:49.908599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:49.908974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9feda6-00d1-4a13-8982-e90067c769ea None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:49.910447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9feda6-00d1-4a13-8982-e90067c769ea None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-eb9feda6-00d1-4a13-8982-e90067c769ea None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:49.922239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:49.923336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 295/596] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1c48-8a71-44cc-a25f-1d5be181cc65 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:49.929117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1c48-8a71-44cc-a25f-1d5be181cc65 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:49.929959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1c48-8a71-44cc-a25f-1d5be181cc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:49.942239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-312b1c48-8a71-44cc-a25f-1d5be181cc65 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:49.943436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-312b1c48-8a71-44cc-a25f-1d5be181cc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:50.008565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf3fa-47a9-4348-90d4-964df9b4678b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.010414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 296/597] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:50.011239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1c48-8a71-44cc-a25f-1d5be181cc65 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.052387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.053030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.053308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.053581 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.060462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.062081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:50.072986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:50.074025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 302/598] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:50.074938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.125247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1c48-8a71-44cc-a25f-1d5be181cc65 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.142894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:50.144339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:50.162534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.163155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.163434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.163883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.164669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:50.164973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.166422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.179613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 297/599] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:50.179613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.186045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda82fe-601b-44a8-8c24-d294bf33c72e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.222137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.223165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.223442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.223752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/a3b919bcbb104fa3bf11847b6a35a221/roles/be623ba761364dd593a441e1ca9cd988` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.238821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=a3b919bcbb104fa3bf11847b6a35a221, role_id=be623ba761364dd593a441e1ca9cd988)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.240311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.254383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 303/600] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:50 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/a3b919bcbb104fa3bf11847b6a35a221/roles/be623ba761364dd593a441e1ca9cd988 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:50.255202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.304773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cced2a8-db12-4c38-8ec8-4e83815b84d1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.341956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.342703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.343041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.343373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.344253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:50.344623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.346192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.370292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 298/601] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Aug 22 05:25:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:50.370292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.384530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595165a-443b-4426-a312-e23dc04393a3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.407480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:50.410347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:50.435082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.435765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.436045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.436316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.437035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:50.437335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.438793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.451991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 304/602] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:50.452679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.497797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e21ff4-5cb2-48df-b0ee-8ecd80d9e173 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.533280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.533959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.534240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.534514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/69d9ee15b5fa43248fbdca69c3c91a6b/roles/be623ba761364dd593a441e1ca9cd988` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.549341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=69d9ee15b5fa43248fbdca69c3c91a6b, role_id=be623ba761364dd593a441e1ca9cd988)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.550803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.559439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cea38-e816-4dad-b4a0-f120d4cb3091 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.567065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 299/603] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:50 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/69d9ee15b5fa43248fbdca69c3c91a6b/roles/be623ba761364dd593a441e1ca9cd988 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:50.568112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.615428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.616125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.616406 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.616680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.638363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:50.644175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:50.645321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:50.646329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:50.653908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 305/604] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:50.655069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.689538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:50.704688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:50.712544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f979a-d019-4835-81b1-ef700b5b6c24 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.760742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.761394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.761693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.761939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.775331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.776706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.792424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8dcc1-53ba-447c-8b6c-0471e7ca239e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:50.794892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:50.796038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:50.797074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:50.797074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:50.797074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:50.797074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 300/605] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:50.797310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.845037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.845977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.846417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.846827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/3fcf79bef8d542d9adbfce89931032c0/roles/be623ba761364dd593a441e1ca9cd988` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.873181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=3fcf79bef8d542d9adbfce89931032c0, role_id=be623ba761364dd593a441e1ca9cd988)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.874644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.909426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 306/606] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:50 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/3fcf79bef8d542d9adbfce89931032c0/roles/be623ba761364dd593a441e1ca9cd988 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:50.910124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:50.939015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0d4b8-6fdf-4d29-960c-f1c8a94dc98e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:50.982506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:50.983458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:50.983907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:50.984465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:50.985402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:50.986072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:50.987653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:50.990484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:51.002395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 301/607] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:51.007373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:51.109235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:51.137086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc45b29-9ae2-4512-a6a3-86d8489ad01e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:51.141818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289211cb-3096-443a-be15-ef14a30bad49 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:51.177733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:51.178697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:51.179111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:51.179585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:51.186231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:51.186856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:51.187130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:51.187397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:51.188432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:51.188432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:51.189742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:51.198298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:51.203973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:51.216534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 307/608] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:51.223510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:51.242591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:51.244885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:51.246060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:51.246060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:51.246060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:51.246060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 302/609] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:51.249374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:51.340200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:51.389185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1189221-b813-4e9b-83de-b9b4dccbef8e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:51.429823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:51.432741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:51.433115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:51.433518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:51.434282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:51.435420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:51.438117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:51.459969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed5c97-eb13-49d1-946c-e8deef2bd175 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:51.465184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 303/610] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:51.470674 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:51.495712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:51.496504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:51.496878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:51.497242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:51.504684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:51.506616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:51.518881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:51.520010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:51.521110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:51.523593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 308/611] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:51.531046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:51.546529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:51.646361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbfdfc-e840-4816-886d-8c474ca0f95b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:51.699429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:51.700182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:51.700551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:51.701727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:51.704048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:51.704465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:51.707222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:51.713039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0bba-2d9d-44ca-af2a-293ee73f8ddb None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:51.733789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 309/612] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:51.735707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:51.741682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:51.785662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:51.786353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:51.786633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:51.786902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:51.800153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:51.801835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:51.827690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:51.828689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:51.829611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 304/613] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:51.837333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:51.860007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:51.911196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379ae2e-3180-46ea-9dd9-d347a7417c83 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:51.987937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:51.988538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:51.988811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:51.989077 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/0a5c594aafc14c1e86ffa9702f9be964/roles/be623ba761364dd593a441e1ca9cd988` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.017171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=0a5c594aafc14c1e86ffa9702f9be964, role_id=be623ba761364dd593a441e1ca9cd988)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:52.018764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:52.044927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 310/614] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:51 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/0a5c594aafc14c1e86ffa9702f9be964/roles/be623ba761364dd593a441e1ca9cd988 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:25:52.046898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1eeb6-2e36-455f-bc4c-b1c4a982ce87 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:52.104494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee643d3c-edcf-474c-91e7-5beb67fb33b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:52.105145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee643d3c-edcf-474c-91e7-5beb67fb33b0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.105419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee643d3c-edcf-474c-91e7-5beb67fb33b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.105687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee643d3c-edcf-474c-91e7-5beb67fb33b0 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.106443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee643d3c-edcf-474c-91e7-5beb67fb33b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:52.106744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee643d3c-edcf-474c-91e7-5beb67fb33b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:52.108448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee643d3c-edcf-474c-91e7-5beb67fb33b0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:52.135785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 305/615] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 22 05:25:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:52.141377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:52.270362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ed1a-282e-4dd6-87e9-eff04e3dd121 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:52.296085 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:52.316784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:52.317426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.317701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.317967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.331415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:52.332929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:52.348893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:52.349876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:52.350753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:52.350753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:52.350753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:52.351506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e462b61-a8dd-4ab7-a09c-d2f57f87f65b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.351980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e462b61-a8dd-4ab7-a09c-d2f57f87f65b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.352154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e462b61-a8dd-4ab7-a09c-d2f57f87f65b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.352909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 311/616] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:52.353651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 306/617] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:52.358325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:52.364637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b867981-7140-4235-a12e-8271d7b3e465 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.365199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b867981-7140-4235-a12e-8271d7b3e465 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.365692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b867981-7140-4235-a12e-8271d7b3e465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.389615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-0b867981-7140-4235-a12e-8271d7b3e465 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:52.390901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0b867981-7140-4235-a12e-8271d7b3e465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:52.418116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:52.520883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 307/618] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:52.533342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed5c71-656a-4d05-93bd-86bece4aae57 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:52.572419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e76b01-2d5b-4431-8312-17adf3274674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:52.573349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e76b01-2d5b-4431-8312-17adf3274674 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.573802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e76b01-2d5b-4431-8312-17adf3274674 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.574243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e76b01-2d5b-4431-8312-17adf3274674 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.575088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e76b01-2d5b-4431-8312-17adf3274674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:52.575557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e76b01-2d5b-4431-8312-17adf3274674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:52.577133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e76b01-2d5b-4431-8312-17adf3274674 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:52.598361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 312/619] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:52.604588 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b867981-7140-4235-a12e-8271d7b3e465 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:52.734696 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b867981-7140-4235-a12e-8271d7b3e465 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:52.767699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d070b6-b9d6-48d4-8baf-923a423c0bbf None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.768053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d070b6-b9d6-48d4-8baf-923a423c0bbf None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.768248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d070b6-b9d6-48d4-8baf-923a423c0bbf None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.770008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 313/620] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:52.775074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4d7c7-fda3-471c-9e6d-b056c84528b2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.775293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4d7c7-fda3-471c-9e6d-b056c84528b2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.775565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4d7c7-fda3-471c-9e6d-b056c84528b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.781248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:52.782751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:52.783928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:52.784406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:52.799654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:52.801475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:52.806842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-def4d7c7-fda3-471c-9e6d-b056c84528b2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:52.808072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-def4d7c7-fda3-471c-9e6d-b056c84528b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:52.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:52.815406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:52.816334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:52.818719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 308/621] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:52.828536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:52.835990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:52.857789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:52.910841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 314/622] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:52.912063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4d7c7-fda3-471c-9e6d-b056c84528b2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:52.977697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eccd17-8efe-4d20-9302-e526ca9d72e3 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.013215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4d7c7-fda3-471c-9e6d-b056c84528b2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.050422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.051441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.051943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.052982 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.062351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.063280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.063797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.064274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.066126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:53.066797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:53.069359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:53.076416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:53.085398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:53.086546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:53.087371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:53.096415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 315/623] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:53.099286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 309/624] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:53.101225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:53.105504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:53.137875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:53.213796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:53.238957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b788201-5d1b-4aca-9732-470bc7459592 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.277991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.278667 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.278942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.279209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/b63e2db3e07c4471be6928ff31334816/roles/be623ba761364dd593a441e1ca9cd988` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.286631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca8e2b-d231-4c1f-b2a7-4f0e00a40079 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.299772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=b63e2db3e07c4471be6928ff31334816, role_id=be623ba761364dd593a441e1ca9cd988)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:53.301319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:53.323694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 310/625] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:25:53 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/b63e2db3e07c4471be6928ff31334816/roles/be623ba761364dd593a441e1ca9cd988 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:25:53.323694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:53.333798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.335408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.336157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.338754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.339747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:53.340215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:53.341831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:53.368497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 316/626] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:53.374634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:53.512264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:53.522778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0f4dc-e038-401e-9eb9-4698749546c0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.542035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b70a6f-ed6a-40f7-a343-c85b669f0bcf None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.568847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:53.582083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc9916-9a85-4b66-9b27-bb28bf08098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.583040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc9916-9a85-4b66-9b27-bb28bf08098a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.583903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc9916-9a85-4b66-9b27-bb28bf08098a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.584253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc9916-9a85-4b66-9b27-bb28bf08098a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.596340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f77551-7a44-4a69-94fe-9ad9c96a1d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.597848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77551-7a44-4a69-94fe-9ad9c96a1d22 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.597965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc9916-9a85-4b66-9b27-bb28bf08098a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:53.598370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77551-7a44-4a69-94fe-9ad9c96a1d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.598782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77551-7a44-4a69-94fe-9ad9c96a1d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8dfc9916-9a85-4b66-9b27-bb28bf08098a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:53.603705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:53.604851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:53.605781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:53.606494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 311/627] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:53.607280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905bb25-6f41-42d7-9108-307e0e4801b7 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.612134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905bb25-6f41-42d7-9108-307e0e4801b7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.614225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905bb25-6f41-42d7-9108-307e0e4801b7 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.620053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f77551-7a44-4a69-94fe-9ad9c96a1d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:53.620053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f77551-7a44-4a69-94fe-9ad9c96a1d22 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:53.620434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 312/628] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:53.621504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905bb25-6f41-42d7-9108-307e0e4801b7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-a1f77551-7a44-4a69-94fe-9ad9c96a1d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:53.635276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:53.636528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:53.637722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:53.637722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:53.637722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:53.638950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 317/629] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:53.640269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddef268-4bbc-4c0b-a1c8-a4e39aba690c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.640657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddef268-4bbc-4c0b-a1c8-a4e39aba690c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.641012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddef268-4bbc-4c0b-a1c8-a4e39aba690c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.659280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3ddef268-4bbc-4c0b-a1c8-a4e39aba690c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:53.660650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3ddef268-4bbc-4c0b-a1c8-a4e39aba690c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:53.739616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 318/630] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:53.739616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddef268-4bbc-4c0b-a1c8-a4e39aba690c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:53.748332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905bb25-6f41-42d7-9108-307e0e4801b7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.790141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.791056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.791473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.792619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.793467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:53.798627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:53.800230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:53.813055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 313/631] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:53.820336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:53.871927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:53.896515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddef268-4bbc-4c0b-a1c8-a4e39aba690c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.917419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:53.967066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:53.975597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:53.975597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:53.975597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:53.975597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:53.975597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:53.975597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:53.978689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a1cd-1ea9-464f-ad43-49f0f1a15639 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:53.983459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 319/632] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:53.993239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.018900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.019913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.020407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.020811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.032526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.034188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:54.049097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:54.050263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:54.051375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:54.051375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:54.051375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:54.053148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 314/633] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:54.060462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.162998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524e1e3-0679-4bb6-8b7c-f946477b5699 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.178451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dd5d7-a757-49b6-b819-98fdf3b874b0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.184666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:54.196046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:54.211357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.212587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.213279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.213718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.223248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.223902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.224184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.224370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.224494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.225098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:54.225401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.226604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.226899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:54.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:54.242115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:25:54.243102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:54.243846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 315/634] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:54.244753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.245920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 320/635] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:25:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:54.246937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.356175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84be912-126d-4ee0-baa8-6dfcbb47844c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.371815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555c9b-c7a9-43c1-bf8d-66b6ce2a947f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.401105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.401973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.402251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.402894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.407052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ced23-1302-4990-a0fb-5dcdcac6b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.407726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ced23-1302-4990-a0fb-5dcdcac6b851 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.408012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ced23-1302-4990-a0fb-5dcdcac6b851 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.408287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ced23-1302-4990-a0fb-5dcdcac6b851 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.414894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:54.416078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ced23-1302-4990-a0fb-5dcdcac6b851 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.417540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ced23-1302-4990-a0fb-5dcdcac6b851 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:54.421194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:54.422330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:54.423379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:54.429579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 316/636] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:54.429579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-548ced23-1302-4990-a0fb-5dcdcac6b851 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:54.430433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:54.431547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 321/637] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8132f-bdf9-4bdb-a666-7678526dbaad None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.437181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8132f-bdf9-4bdb-a666-7678526dbaad None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.437979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8132f-bdf9-4bdb-a666-7678526dbaad None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.437979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 322/638] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:54.437979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8132f-bdf9-4bdb-a666-7678526dbaad None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.458716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:54.486961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:54.549431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ee607-7a2d-4eac-9466-c715fea9924a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.554045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8132f-bdf9-4bdb-a666-7678526dbaad None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.590374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.590631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ea2f2-66f6-452f-93a1-13701cf5c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.591005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.591128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ea2f2-66f6-452f-93a1-13701cf5c12d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.591289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.591391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ea2f2-66f6-452f-93a1-13701cf5c12d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.591561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.591684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ea2f2-66f6-452f-93a1-13701cf5c12d None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.592255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:54.592349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ea2f2-66f6-452f-93a1-13701cf5c12d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:54.592553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.592646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ea2f2-66f6-452f-93a1-13701cf5c12d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.593967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.594079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ea2f2-66f6-452f-93a1-13701cf5c12d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.605033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 323/639] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:54.605898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.622471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 317/640] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:54.622471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8a43-8e02-43db-898b-4d5b5c9eea58 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.622471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8a43-8e02-43db-898b-4d5b5c9eea58 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.622471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8a43-8e02-43db-898b-4d5b5c9eea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.637745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-78bb8a43-8e02-43db-898b-4d5b5c9eea58 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:54.638936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-78bb8a43-8e02-43db-898b-4d5b5c9eea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:54.705785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 318/641] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:54.706551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8a43-8e02-43db-898b-4d5b5c9eea58 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.726155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:54.731720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff15f03-5d22-4403-92a6-a17b725b30b6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.734957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:54.783091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.784249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.785053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.785602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.786628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:54.787344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.790063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.813619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 324/642] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:54.814515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.846418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8a43-8e02-43db-898b-4d5b5c9eea58 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.893904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.894861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.895301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.895798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:54.907739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:54.910403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:54.931360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:54.932662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:54.933738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:54.933738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:54.933738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:54.935792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 319/643] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:54.936254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f96c-b8e5-414d-b4bc-0001d3e01e19 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:54.936897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:54.975207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:54.975944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:54.976375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:54.976903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.011961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.015080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.053350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.054873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.055991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd43f07-9a05-49d8-838b-94cf83f3e4bf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.090640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.091409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.091859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.092234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.163191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 325/644] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:55.163191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.167978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.169562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.226833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.233700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.268025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95828d5-99aa-4854-9c82-d08bab6acedb None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.281649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 320/645] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:55.282771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.307096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.307741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.308017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.308282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.308924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:55.309221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.310604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.326469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 326/646] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:55.326469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.391842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab18a1-4ade-4ab7-ae10-101c815e4df6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.425851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ede2-4eff-4053-b1c4-c5c315a7ef5b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.430055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.430686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.430964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.431231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.441032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:55.445914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:55.447510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 321/647] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:55.449408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.453906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.464587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.465228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.465504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.465770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.473247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.476993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.478400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:55.492316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:55.493267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:55.494179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:55.494179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:55.494179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:55.494179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 327/648] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:55.494400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.551281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f020292-67a0-4da5-b662-022e3e6c1da9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.584560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a280c-fdfa-44ed-b8c1-d68d88488a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.585188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a280c-fdfa-44ed-b8c1-d68d88488a93 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.585501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a280c-fdfa-44ed-b8c1-d68d88488a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.585762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a280c-fdfa-44ed-b8c1-d68d88488a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.591226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ebb79-bf61-4071-ae18-a10b1f2363d8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.595938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a280c-fdfa-44ed-b8c1-d68d88488a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.597776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a280c-fdfa-44ed-b8c1-d68d88488a93 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-673a280c-fdfa-44ed-b8c1-d68d88488a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:55.612002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:55.613108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:55.614135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:55.614135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:55.614135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:55.614135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 322/649] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:55.614396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1cb96-3faa-4d00-90a8-d610fee31ad4 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.614537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1cb96-3faa-4d00-90a8-d610fee31ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.614803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1cb96-3faa-4d00-90a8-d610fee31ad4 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.616052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 323/650] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:25:55.616779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1cb96-3faa-4d00-90a8-d610fee31ad4 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.631231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.631913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.632192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.632493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.640156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.641629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:55.652639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:55.653596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 328/651] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:55.654948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.689886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.692149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.715358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1cb96-3faa-4d00-90a8-d610fee31ad4 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.747824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.748417 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.748699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.748971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.749635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:55.749935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.751311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.757077 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c66c5d-3753-4c0f-ac01-efc71db36227 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.759066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 324/652] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:55.759909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.796033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd62ebf-2a76-472b-b4ef-04ed68e9b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.796643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62ebf-2a76-472b-b4ef-04ed68e9b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.796923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62ebf-2a76-472b-b4ef-04ed68e9b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.797192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62ebf-2a76-472b-b4ef-04ed68e9b9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.797837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd62ebf-2a76-472b-b4ef-04ed68e9b9f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:55.798534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd62ebf-2a76-472b-b4ef-04ed68e9b9f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.799942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd62ebf-2a76-472b-b4ef-04ed68e9b9f4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.811547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 329/653] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:55.812716 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457754-82ef-4fa6-a55e-38b230de7159 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.813008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457754-82ef-4fa6-a55e-38b230de7159 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.813273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457754-82ef-4fa6-a55e-38b230de7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.831003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-59457754-82ef-4fa6-a55e-38b230de7159 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:55.832197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-59457754-82ef-4fa6-a55e-38b230de7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:55.857068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4cb5a-5c46-41e4-b4b1-4c4fccd34c11 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:55.891991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:55.892603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:55.892888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:55.893162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:55.893815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:55.894106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:55.895503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:55.898512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:55.898777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 330/654] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:55.899819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457754-82ef-4fa6-a55e-38b230de7159 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.913386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 325/655] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:55.914258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:55.930491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.011543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457754-82ef-4fa6-a55e-38b230de7159 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.027547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e08661-97f2-4d69-9a59-c99dab8b06d6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.047433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.048194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.048474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.048746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.049478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:56.049782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.051373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.062379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.063016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.063294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.063566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.074466 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 331/656] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:56.075331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.075501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.076963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:56.092313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:56.093479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:56.094571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:56.094571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:56.094571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:56.098413 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 326/657] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:56.099366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.149228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.163775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.182737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089b1a9-4d4a-4d1b-88cf-a0bec912956f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.203276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c391ba-a5b2-4b41-9b5d-f2a27a80f42a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.215301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.216169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.216447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.216715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.223491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.224960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:56.235887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:56.236816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:56.237690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 332/658] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:56.238197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.238197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.238351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.238422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.238422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.248401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.249854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:56.261390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:56.262471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:56.263371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:56.264062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 327/659] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:56.264860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.341137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3bbc2-dc75-422c-a283-61489464adc1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.363899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc08084-dc2a-4d20-a4d1-19be6752cdec None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.372684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.374374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.374988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.375267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.375541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.387498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.397068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.397680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.398093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.398523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.399217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:56.399691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.401093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.412756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 328/660] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:56.413752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.442901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.444376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.511007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295c63a-757f-4a28-a9a5-e13e2ebc2754 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.545354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.545905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.546187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.546424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.549023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 333/661] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:56.549855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.557494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:25:56.562223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:56.563160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:56.564001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 329/662] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:56.564692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.592722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.596658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.652535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc007157-dd31-461d-960e-bc2f602ed1f7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.667006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dfa1d-0df9-4e6f-b786-dd828fd88397 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.687084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.687809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.688111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.688378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.689057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:56.689363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.690794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.701133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.701740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.702017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.702329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.702598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 334/663] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:56.703022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:56.703331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.703470 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.704791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.717747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 330/664] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:56.718541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.803654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.805770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5d9a0-dc8c-488c-9f43-ede7e117e4c6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.807379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:56.816369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77617c24-6859-44dc-a247-c6441e3bbd98 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.841768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.842407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.842682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.842952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.849759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:56.850391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:56.850673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:56.850849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.850957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:56.852988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.861894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:56.863350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:56.864476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:56.865382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:56.866297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 335/665] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:56.866978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:56.882952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:56.884376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:56.885570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:56.885570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:56.885570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:56.885570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 331/666] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:56.885570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:56.978025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642415f1-4687-44db-a0ba-74ae012f6079 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:56.987158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884fd83-1880-431f-a38f-16dc60366666 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.017538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.018156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.018437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.018745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.019387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:57.019815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.021269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.026655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.027270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.027552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.027854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.028582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=a3b919bcbb104fa3bf11847b6a35a221) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:57.028889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.029366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 336/667] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:57.030143 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.030383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.038232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 332/668] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:56 2022] GET /identity/v3/role_assignments?user.id=a3b919bcbb104fa3bf11847b6a35a221&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:57.039058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.051154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.064459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.150879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979f212-4a5a-40b5-97da-b809dfb590f3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.162630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd76bb-a120-4797-b820-181b1c42800f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.191459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.192117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.192427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.192713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.193449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=69d9ee15b5fa43248fbdca69c3c91a6b) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:57.193750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.195308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.201777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 337/669] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/role_assignments?user.id=69d9ee15b5fa43248fbdca69c3c91a6b&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:57.202609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.206811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.207456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.207767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.208037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.208797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:57.209101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.210636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.231614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 333/670] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:57.232607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.328078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.341254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.363646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3998502-3d23-4fb9-b1e4-42c027ec43a7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.379279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a904be-cf88-4749-b274-04d545b263ec None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.411041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.412081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.412379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.413156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.413782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:57.414092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.415480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.421100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.421284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.421481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.421694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.433378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.434879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.435671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 338/671] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:57.439746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:57.459894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:57.461308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:57.462187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:57.462187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:57.462187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:57.463685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 334/672] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:57.464642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.568088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3413dc3-a56d-49e0-a2c1-6d8c4dc0c62d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.588285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffc53b-f94f-475f-9bb9-5d8f5c6ebebf None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.604838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.623726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.625934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.626217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.626493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.628736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.629611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.630071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.630517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.640815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.642576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.654143 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:57.655704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:57.656592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:57.657473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:57.659562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 335/673] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:57.660655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.709872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.711442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.764822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c5628-f443-43e2-a243-a3f6f4bb6a8e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.801608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:57.802940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:57.803438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:57.803948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:57.804972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:57.805932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:57.807823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:57.823259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 339/674] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:25:57.824375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.824503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 336/675] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:57.830225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:57.868225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.891821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:57.957595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560c98-b9b6-410f-9b31-e34bbd423e4b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:57.973225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e016602-ff0b-4896-8b03-c7baa6150854 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.001455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.002709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.003179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.003872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.014572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.015432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.016171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.016547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.016785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.016995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.017791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:58.018198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.019809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:25:58.031555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:58.032735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:25:58.033850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:58.034488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 340/676] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:58.035377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.036251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 337/677] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:58.039798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.148398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b937bca-1368-4c84-8c80-61f836b915e9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.158327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:58.164656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:58.174086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14d82e-1c54-4ed7-8488-8b09f2bdaa67 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.188781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.189554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.189951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.190317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.202689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.204536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.218681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.219509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.219980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.220423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:58.221003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:58.222122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:58.223567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:58.223567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:58.223567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:58.224186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=3fcf79bef8d542d9adbfce89931032c0) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:58.224628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.226277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.229840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 341/678] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:58.230543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.233682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 338/679] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/role_assignments?user.id=3fcf79bef8d542d9adbfce89931032c0&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:58.235602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.374134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186e6b5-15ef-4417-8bfe-9d88e0a1879e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.376764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dd0d4-635b-4882-9e8b-df2622c7be68 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.416873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.417765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.418197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.418604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.419409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:58.419898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.422239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.437474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 339/680] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:58.448008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.448749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.449384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.449662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.449933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.450686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:58.450992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.452649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.469604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:58.477542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 342/681] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:58.483421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.508356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:58.583047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cea8e-40bb-42ba-86ff-91edd9542a01 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.620828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b523048-7296-40ae-bcb6-0c7763a729a6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.621199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.621852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.622156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.622525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.630888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.632496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:58.645474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:58.646458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:25:58.647325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 340/682] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:58.655813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.664112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.664740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.665018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.665293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.780163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.781538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.788312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:58.812559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:58.829114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65357a6-d557-4863-bba6-019e666730c8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:58.866344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:58.867115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:58.867392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:58.867685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:58.868398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:58.868703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:58.870345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:58.887783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 341/683] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:25:58.889209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:58.982795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 343/684] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 05:25:58.998813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72234f9-fbe3-461e-80f6-ea0807a3a27e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:59.000074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:59.034110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8653b1e-f298-4185-85ce-9567d9e9856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:59.034642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8653b1e-f298-4185-85ce-9567d9e9856e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.034927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8653b1e-f298-4185-85ce-9567d9e9856e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.035199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8653b1e-f298-4185-85ce-9567d9e9856e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.036062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8653b1e-f298-4185-85ce-9567d9e9856e None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=0a5c594aafc14c1e86ffa9702f9be964) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:59.036374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8653b1e-f298-4185-85ce-9567d9e9856e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:59.037825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8653b1e-f298-4185-85ce-9567d9e9856e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:59.049163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 342/685] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/role_assignments?user.id=0a5c594aafc14c1e86ffa9702f9be964&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:59.199645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cdad5-8476-491d-9f05-49712f4d8a55 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:59.224869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:59.238080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:59.238741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.239027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.239691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.251806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:59.253278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:59.269859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b861e-3e38-45c6-9625-b4557f05b67b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:25:59.271444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:59.272592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:25:59.274019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:25:59.274019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:25:59.274019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:59.274214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b861e-3e38-45c6-9625-b4557f05b67b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.274214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b861e-3e38-45c6-9625-b4557f05b67b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.274783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 344/686] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:25:59.276021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 343/687] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:25:59.280448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:59.282083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe3f2f-cf6e-46ea-bf5d-ba57007e4d01 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.282572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe3f2f-cf6e-46ea-bf5d-ba57007e4d01 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.283269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe3f2f-cf6e-46ea-bf5d-ba57007e4d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.293849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:59.311758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-affe3f2f-cf6e-46ea-bf5d-ba57007e4d01 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:59.313319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-affe3f2f-cf6e-46ea-bf5d-ba57007e4d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:59.392992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57f3cf-7003-4196-9cca-d13f3ff8694f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:59.404830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 344/688] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:25:59.453261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:59.454159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.454573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.454977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.455785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:59.456248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:59.457754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:59.477022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e264466-2712-4d48-8cd3-21b1497086f1 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.477306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e264466-2712-4d48-8cd3-21b1497086f1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.477580 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e264466-2712-4d48-8cd3-21b1497086f1 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.477946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 345/689] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:59.479200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 345/690] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:25:59.483328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:59.484478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581ab7c-b00f-4d1d-906f-368953575271 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.484766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581ab7c-b00f-4d1d-906f-368953575271 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.485041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581ab7c-b00f-4d1d-906f-368953575271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.516725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e581ab7c-b00f-4d1d-906f-368953575271 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:25:59.517948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e581ab7c-b00f-4d1d-906f-368953575271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:25:59.538165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:59.645711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d56d3-fe23-4d61-a523-8fc7b2f8fe7c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:59.677734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 346/691] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:25:59.693206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:59.693868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.694156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.694431 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.708739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:59.710609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:25:59.728515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:25:59.729604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:25:59.730527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 346/692] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:25:59.735629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581ab7c-b00f-4d1d-906f-368953575271 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:59.744746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:25:59.843877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:59.885901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:25:59.913788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395101c-19dc-462d-b246-4b6c7d19f9a4 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:59.920165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581ab7c-b00f-4d1d-906f-368953575271 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:25:59.959189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:25:59.959852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:25:59.960129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:25:59.960605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:25:59.961139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:25:59.961440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:25:59.962885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:25:59.976061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 347/693] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:25:59.981670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.030795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.031444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.031761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.032032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.042603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:00.047311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:00.049475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 347/694] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:00.051514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.105766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54199-6a5d-48df-8c57-029977f1cdb4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:00.135995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:00.153509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.154307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.154939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.155235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.156250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=b63e2db3e07c4471be6928ff31334816) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:00.156557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:00.157945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e482d-9420-47c2-b57b-892eefc735de None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:00.158078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:00.165422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 348/695] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:25:59 2022] GET /identity/v3/role_assignments?user.id=b63e2db3e07c4471be6928ff31334816&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:00.166750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.196045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.196885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.197271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.197612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.211779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:00.213384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:00.230203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:00.232189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:00.234045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:00.234045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:00.234045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:00.234045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 348/696] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:00.242052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.251198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:00.331979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d18ebe-03e5-4b1e-b9b3-a6b272245b04 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:00.388538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.389265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.389555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.389819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.390496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:00.390794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:00.392223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:00.401587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 349/697] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:00.407709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.427935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3155ee-72a4-49e4-b3f6-73f9317dcd1e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:00.462614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.463611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.463863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:00.464305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.464763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.465626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:00.466111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:00.467695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:00.481986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 349/698] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:00.487792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.529353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d915c8e-8985-45c3-bd45-afc4fe9294ed None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:00.576683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.577317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.577634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.577901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.579322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:00.591062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:00.592677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:00.607756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:00.608797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:00.609695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:00.609695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:00.609695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:00.609695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 350/699] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:00.614816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.626660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453034-b5bc-42cc-9074-89cdd1e4c921 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:00.663755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6f52-d72d-4197-87c3-46ea8be3677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.664936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6f52-d72d-4197-87c3-46ea8be3677d None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.665478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6f52-d72d-4197-87c3-46ea8be3677d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.666174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6f52-d72d-4197-87c3-46ea8be3677d None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.667777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6f52-d72d-4197-87c3-46ea8be3677d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:00.670067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6f52-d72d-4197-87c3-46ea8be3677d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:00.697865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 350/700] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:26:00 2022] POST /identity/v3/users => generated 254 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:00.700921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3ad2-7c7c-4f27-89cb-fc562c2b8924 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.701740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3ad2-7c7c-4f27-89cb-fc562c2b8924 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.702220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3ad2-7c7c-4f27-89cb-fc562c2b8924 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.705293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 351/701] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:00.714080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242cc7d-b996-455b-9602-260931ec8cd2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.714080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242cc7d-b996-455b-9602-260931ec8cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.714080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242cc7d-b996-455b-9602-260931ec8cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.741396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:00.745493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c242cc7d-b996-455b-9602-260931ec8cd2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:00.747793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bba159-2689-426b-b0aa-e28ab610065e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:00.752810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c242cc7d-b996-455b-9602-260931ec8cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:00.850628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:00.850628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:00.850628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:00.850628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:00.851272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:00.851272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:00.853792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:00.861280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 352/702] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:26:00.873879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 351/703] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:00.879014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242cc7d-b996-455b-9602-260931ec8cd2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:00.948576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:01.008185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242cc7d-b996-455b-9602-260931ec8cd2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:01.052238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:01.052888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.053165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.053441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.064063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:01.065527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:01.081678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:01.082661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:01.083694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:01.085756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 353/704] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:26:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:01.090683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:01.195468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:01.230425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282813-1781-4a23-8f30-e4a400c92094 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:01.285904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:01.286529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.286807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.287079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.287849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:01.288157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:01.289959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:01.309794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 352/705] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:26:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:01.315204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:01.344051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:01.424180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6c4cc-1550-451b-958e-aa2a123a637b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:01.464624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b70c2-e696-4cb9-8af9-cfe539253daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:01.465762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b70c2-e696-4cb9-8af9-cfe539253daf None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.466173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b70c2-e696-4cb9-8af9-cfe539253daf None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.466446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b70c2-e696-4cb9-8af9-cfe539253daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.473365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3447-e60d-4a73-a89b-a1a1f03d0bf2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:01.484202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:01.511426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26315e15-4c12-444a-8418-71f98e182683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:01.512209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26315e15-4c12-444a-8418-71f98e182683 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.512555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26315e15-4c12-444a-8418-71f98e182683 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.512888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26315e15-4c12-444a-8418-71f98e182683 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.513548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26315e15-4c12-444a-8418-71f98e182683 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:01.515503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26315e15-4c12-444a-8418-71f98e182683 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-26315e15-4c12-444a-8418-71f98e182683 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:01.531008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:01.532320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:01.533632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 354/706] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Aug 22 05:26:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:01.534625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd5888-16b7-42af-adc4-b52f824319a5 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.537352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd5888-16b7-42af-adc4-b52f824319a5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.537352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd5888-16b7-42af-adc4-b52f824319a5 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.539702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 355/707] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:01.540578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd5888-16b7-42af-adc4-b52f824319a5 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:01.554957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b70c2-e696-4cb9-8af9-cfe539253daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:01.557329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b70c2-e696-4cb9-8af9-cfe539253daf None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:01.580706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:01.667391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd5888-16b7-42af-adc4-b52f824319a5 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:01.677526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 353/708] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 05:26:01.679152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5229c-51dc-488e-9418-e00f87dd4448 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.679789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5229c-51dc-488e-9418-e00f87dd4448 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.680069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5229c-51dc-488e-9418-e00f87dd4448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.703021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-33b5229c-51dc-488e-9418-e00f87dd4448 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:01.704830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-33b5229c-51dc-488e-9418-e00f87dd4448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:01.715860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:01.717275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.717756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.718227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.719102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:01.719682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:01.721331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:01.727912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 356/709] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 22 05:26:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:01.729016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:01.764870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:01.787332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 354/710] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:26:01.788302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5229c-51dc-488e-9418-e00f87dd4448 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:01.831002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc314e-c451-49c2-840f-525f4d65967b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:01.883925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:01.893855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:01.894806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:01.895305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:01.895794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:01.911625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:01.916308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:01.942907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:01.944023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:01.944984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:01.944984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:01.944984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:01.948354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 357/711] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:01.954519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:01.982049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5229c-51dc-488e-9418-e00f87dd4448 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:02.047984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:02.048523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.048796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.049062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.049697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:02.049992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:02.051370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:02.062416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 355/712] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 22 05:26:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:02.111065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:02.124619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:02.145656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a80b3-800f-4bb1-84bc-84c4a5d737ea None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:02.218103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:02.218735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.219008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.219278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.219969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:02.220269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:02.221665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:02.236011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 358/713] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:02.241815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:02.299380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1c937-6619-459f-b10e-8a088c72e7b6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:02.312931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:02.335035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:02.335966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.336314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.336649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] PATH_INFO: `/v3/services/network` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.337306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:02.338821 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:02.350982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:02.352026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:02.352937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 356/714] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:02 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:02.358108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:02.359698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f44a2-d3fc-4471-90f8-e5f0ca1c88cd None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:02.395524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:02.396223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.396556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.396830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.417111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:02.418605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:02.445128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:02.446069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:02.446940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 359/715] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:02.456616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:02.463245 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344cafc-7b1a-41fb-8e6c-abeb5b670f47 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:02.478568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:02.501369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:02.502049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.502328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.502600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.503277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] RBAC: Adding query filter params (name=network) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:02.503608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:02.505159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:02.510633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 357/716] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:02.516677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:02.567301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:02.637687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd4cc2-7ca7-4171-9d20-b0f31693e390 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:02.676985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c6190-4468-4e77-bba4-afd56017490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:02.678288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c6190-4468-4e77-bba4-afd56017490c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.678772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c6190-4468-4e77-bba4-afd56017490c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.679343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c6190-4468-4e77-bba4-afd56017490c None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.681099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c6190-4468-4e77-bba4-afd56017490c None admin] RBAC: Adding query filter params (type=network) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:02.681626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c6190-4468-4e77-bba4-afd56017490c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:02.683683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c6190-4468-4e77-bba4-afd56017490c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:02.691164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 358/717] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:02.695723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a38e7-a481-4976-8692-62f671af835a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:02.697153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc108-3db1-45e5-9010-4d782db61811 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.697436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc108-3db1-45e5-9010-4d782db61811 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.697705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc108-3db1-45e5-9010-4d782db61811 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.699202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 359/718] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:02.711494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b46874-1550-4944-b5fd-ff84a5e2288b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.711878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b46874-1550-4944-b5fd-ff84a5e2288b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.712066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b46874-1550-4944-b5fd-ff84a5e2288b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.739336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f1b46874-1550-4944-b5fd-ff84a5e2288b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:02.740568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f1b46874-1550-4944-b5fd-ff84a5e2288b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:02.783640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:02.784339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:02.784541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:02.784807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:02.785514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:02.785816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:02.787297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:02.817982 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 360/719] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:02.824895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:02.873470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 360/720] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:26:02.934415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:02.971097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fbcd0-9e00-4a94-8559-83b346478475 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:03.021063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d190-7de5-4300-994d-e5506afd70f5 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:03.021416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d190-7de5-4300-994d-e5506afd70f5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:03.021629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d190-7de5-4300-994d-e5506afd70f5 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:03.024711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 361/721] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:03.029557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf39bd-c197-485c-8871-894f2384880e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:03.029898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf39bd-c197-485c-8871-894f2384880e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:03.030111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf39bd-c197-485c-8871-894f2384880e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:03.050065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5ecf39bd-c197-485c-8871-894f2384880e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:03.051306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5ecf39bd-c197-485c-8871-894f2384880e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:03.134352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:03.148450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:03.149015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:03.149289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:03.149541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:03.150284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=62d5a029139c4245a36acda923371996, user.id=a3b919bcbb104fa3bf11847b6a35a221) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:03.150588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:03.152094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:03.159758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 361/722] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:02 2022] GET /identity/v3/role_assignments?user.id=a3b919bcbb104fa3bf11847b6a35a221&scope.project.id=62d5a029139c4245a36acda923371996&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:03.209731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 362/723] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 05:26:03.283176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:03.408813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1e6ee-9ec9-4851-b504-da0461961116 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:03.442452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:03.449238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:03.449909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:03.450183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:03.450454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:03.460928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:03.465521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:03.466537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:03.467440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 362/724] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:03.472975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf39bd-c197-485c-8871-894f2384880e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:03.588486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:03.604113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf39bd-c197-485c-8871-894f2384880e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:03.686223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:03.686758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:03.687033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:03.687301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:03.687995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:03.688292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:03.689676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:03.704796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 363/725] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:03.710511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:03.740045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23640feb-9706-43d1-9ef9-723018ca370f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:03.761195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:03.824795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:03.825518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:03.825710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:03.825975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:03.842213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3921-33e6-41bd-a3b1-b57bf764b1f8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:03.854675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:03.859331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:03.860384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:26:03.861228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:03.863704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 363/726] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:26:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:03.873481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:03.902338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:03.903713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:03.904500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:03.916198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:03.934725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:03.936314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:03.942864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:03.966229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:03.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:03.968209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:03.968209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:03.968209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:03.972578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 364/727] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:03.976555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:03.992686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a78e2c-344a-463a-98c7-530717eecd3a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.076303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:04.076938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.077221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.077503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.078185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:04.078490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:04.079906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:04.095930 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 364/728] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:26:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:04.100581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:04.102425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:04.148470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1f0bc-619d-48d7-968e-2b6114830609 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.219623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:04.221290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.223270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.223716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.224741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:04.226251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:04.229694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:04.256519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 365/729] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:04.261877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:04.288005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49fd99-ef7e-404d-9000-94074765034c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.302530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:04.323749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8ea19-97a8-4cb4-8314-74e551a98960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:04.324538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ea19-97a8-4cb4-8314-74e551a98960 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.324915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ea19-97a8-4cb4-8314-74e551a98960 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.325276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ea19-97a8-4cb4-8314-74e551a98960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.336115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ea19-97a8-4cb4-8314-74e551a98960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:04.337690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ea19-97a8-4cb4-8314-74e551a98960 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-12b8ea19-97a8-4cb4-8314-74e551a98960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:04.352374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:04.353548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:04.354684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:04.354684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:04.354684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:04.355936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 365/730] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:04.357168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfab65-4d5d-4282-9f67-b5b86d3c47b9 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.362075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfab65-4d5d-4282-9f67-b5b86d3c47b9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.362497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfab65-4d5d-4282-9f67-b5b86d3c47b9 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.368358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 366/731] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:04.368358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfab65-4d5d-4282-9f67-b5b86d3c47b9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:04.388452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd11b-418b-40e6-8a5f-ca2e7c06ddb8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.439543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43747ed4-3e17-4e7b-adb0-599883ef28c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:04.440441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43747ed4-3e17-4e7b-adb0-599883ef28c7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.440795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43747ed4-3e17-4e7b-adb0-599883ef28c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.441076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43747ed4-3e17-4e7b-adb0-599883ef28c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.451889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43747ed4-3e17-4e7b-adb0-599883ef28c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:04.453449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43747ed4-3e17-4e7b-adb0-599883ef28c7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:04.455428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-43747ed4-3e17-4e7b-adb0-599883ef28c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:04.469181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:04.470256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:26:04.471222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:04.476969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 366/732] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:04.482653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ab5f-a5a3-4b46-a5b4-ac17058d1e83 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.482947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ab5f-a5a3-4b46-a5b4-ac17058d1e83 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.483226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ab5f-a5a3-4b46-a5b4-ac17058d1e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.494387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfab65-4d5d-4282-9f67-b5b86d3c47b9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.502881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8010ab5f-a5a3-4b46-a5b4-ac17058d1e83 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:04.504103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8010ab5f-a5a3-4b46-a5b4-ac17058d1e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:04.533574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:04.534852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.535415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.535933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.536901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:04.537479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:04.539440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:04.553671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 367/733] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:04.555391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:04.583883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 367/734] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:26:04.583883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ab5f-a5a3-4b46-a5b4-ac17058d1e83 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:04.603611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:04.689446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ab5f-a5a3-4b46-a5b4-ac17058d1e83 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.726044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:04.752346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:04.753005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.753286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.755604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.771298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:04.772893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:04.784636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:04.785649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:04.786575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:04.789063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 368/735] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:26:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:04.791338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:04.833655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f787b1-c0f3-4817-9556-f38d30d9ff87 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.945118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:04.945965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:04.946245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:04.946511 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:04.947331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:04.947655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:04.950900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:04.957663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77752149-1973-490a-b01e-3a50b7a28bb0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:04.984425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 368/736] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 22 05:26:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:04.985170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:05.022040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:05.023007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.023295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.024607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.024607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:05.024809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:05.026206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:05.047686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 369/737] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 22 05:26:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:05.048965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:05.134049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:05.151267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:05.170569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5b6bd-05d9-47a2-bd87-80a707f76de3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:05.200148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30414a03-ba51-42e8-a8e0-346bc7be9aa8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:05.221245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:05.221868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.222496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.222921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.223496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:05.225356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:05.239662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ef79a-452d-4380-be94-d33301ff7956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:05.240224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef79a-452d-4380-be94-d33301ff7956 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.240505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef79a-452d-4380-be94-d33301ff7956 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.240943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ef79a-452d-4380-be94-d33301ff7956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.243675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 369/738] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:04 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:05.243675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:05.326611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ef79a-452d-4380-be94-d33301ff7956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:05.328384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ef79a-452d-4380-be94-d33301ff7956 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:05.383467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:05.408577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44513b-b7b2-4f41-800d-c15bf4f096fa None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:05.449112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb39b83-0ee0-4844-864b-43758cd6b648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:05.449735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39b83-0ee0-4844-864b-43758cd6b648 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.450010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39b83-0ee0-4844-864b-43758cd6b648 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.450283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39b83-0ee0-4844-864b-43758cd6b648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.454796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 370/739] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:26:05.456021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa11805-cd4b-46d7-a879-973fd9699982 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.456939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa11805-cd4b-46d7-a879-973fd9699982 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.456939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa11805-cd4b-46d7-a879-973fd9699982 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.458046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 371/740] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:05.465278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:05.467008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf5930-19ab-4c56-84ef-0deb5cb51652 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.467225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf5930-19ab-4c56-84ef-0deb5cb51652 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.467478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf5930-19ab-4c56-84ef-0deb5cb51652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.495098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c5cf5930-19ab-4c56-84ef-0deb5cb51652 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:05.496269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c5cf5930-19ab-4c56-84ef-0deb5cb51652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:05.527925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb39b83-0ee0-4844-864b-43758cd6b648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:05.529377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb39b83-0ee0-4844-864b-43758cd6b648 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:05.575610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 372/741] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:26:05.575610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf5930-19ab-4c56-84ef-0deb5cb51652 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:05.641551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 370/742] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 05:26:05.642974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533149e1-f773-4a48-b12e-82c4e6b1dfba None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.643315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533149e1-f773-4a48-b12e-82c4e6b1dfba None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.643671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533149e1-f773-4a48-b12e-82c4e6b1dfba None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.645204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 371/743] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:05.646360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533149e1-f773-4a48-b12e-82c4e6b1dfba None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:05.676136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:05.705617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:05.705822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf5930-19ab-4c56-84ef-0deb5cb51652 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:05.744509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44e33c-23e9-404f-a940-1c8848419eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:05.745377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e33c-23e9-404f-a940-1c8848419eea None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.745711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e33c-23e9-404f-a940-1c8848419eea None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.746090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e33c-23e9-404f-a940-1c8848419eea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.757546 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44e33c-23e9-404f-a940-1c8848419eea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:05.759276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44e33c-23e9-404f-a940-1c8848419eea None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-ac44e33c-23e9-404f-a940-1c8848419eea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:05.775303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:05.776533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:05.777640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:05.777640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:05.777640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:05.778606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 373/744] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:05.779602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465b4f-127f-4d1f-a666-c3cf05d56be3 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.781668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465b4f-127f-4d1f-a666-c3cf05d56be3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.781668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465b4f-127f-4d1f-a666-c3cf05d56be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.784046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533149e1-f773-4a48-b12e-82c4e6b1dfba None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:05.804550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c5465b4f-127f-4d1f-a666-c3cf05d56be3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:05.805842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c5465b4f-127f-4d1f-a666-c3cf05d56be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:05.821970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:05.822866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:05.823286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:05.823660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:05.834961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:05.836650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:05.854293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:05.856010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:05.857146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:05.857146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:05.857146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:05.858083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 372/745] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:05.861493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:05.890422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 374/746] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 05:26:05.891606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5465b4f-127f-4d1f-a666-c3cf05d56be3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:05.987966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.003684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b050a1-6efd-4c96-af0c-f597869d35c7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.004475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.064994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.066116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.066584 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.067037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.068380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:06.068897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.071361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.083252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5465b4f-127f-4d1f-a666-c3cf05d56be3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.091516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 373/747] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:06.092811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.147862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.149011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.149500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.150037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.151097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:06.151933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.154787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.172978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 375/748] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:06.174144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.238120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993717b3-59c9-45a7-ad2d-fb072d0f2eab None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.276351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.276906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.277188 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.277464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.278126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.279512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.284951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8db395-54b9-4d4c-8aeb-35c6d4bfac53 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.295625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 374/749] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:06 2022] POST /identity/v3/services => generated 240 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:06.296046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.312060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.325515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.348143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.349211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.349693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.350149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.359243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.361314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:06.376712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:06.377723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:06.378646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:06.381435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 376/750] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:06.382717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.444934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46795d4e-03c0-461d-afab-8f5babccccb7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.487304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.488322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.488600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.488867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.500574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:06.505152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:06.506089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:06.506965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:06.507675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a53fc6-8954-41d4-8420-cce83d5ec4d2 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.507765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 375/751] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:06.507765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.547949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.549031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.549518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.549982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.558171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.559993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:06.572023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:06.572975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:06.573870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:06.576373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 377/752] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:06.581800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.596744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.621456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.637509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54691ad-cfea-4a42-804c-3b3c49f08799 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.673446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.674065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.674347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.674615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.675601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:06.675779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.677211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.690977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 376/753] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:06.692006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.696088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc93889-4f11-4055-8172-13260bfbaee6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.732678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.733762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.734243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.734746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.735673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:06.736194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.737917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.747520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 378/754] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:06.748643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.800559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563d3c-d782-4782-a800-d6f2ac91783b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.834760 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.841178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.841887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.842084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.842376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.843165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:06.843473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.844954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.850599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:06.858509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb12987-9ab8-48c0-be55-cb03631d8529 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:06.864341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 377/755] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:06.864341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:06.907854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:06.908813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:06.909293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:06.909779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:06.910714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=62d5a029139c4245a36acda923371996, user.id=b63e2db3e07c4471be6928ff31334816) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:06.911223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:06.912903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:06.919682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 379/756] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/role_assignments?user.id=b63e2db3e07c4471be6928ff31334816&scope.project.id=62d5a029139c4245a36acda923371996&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:06.920921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:07.067267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d7a4a-61cb-4df5-8272-9c6cd1a75255 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:07.087906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f4624-38e1-4b22-9595-b3a9712871cc None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:07.110320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:07.111298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:07.111832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:07.112303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:07.113233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=9de81bd996d4464d8a299228be695e00) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:07.114405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:07.116107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:07.127114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 380/757] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/role_assignments?user.id=9de81bd996d4464d8a299228be695e00&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:07.146064 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:07.147075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:07.147614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:07.148163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:07.168998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:07.170929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:07.196530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:07.197570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:07.198465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:07.198465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:07.198465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:07.200592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 378/758] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:07.206432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:07.252129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:07.324415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebcb50-bc2f-4b7d-a83b-f98385b19a95 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:07.360454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b7959-12b1-4714-8ff6-9a7e0ed563f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:07.361535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7959-12b1-4714-8ff6-9a7e0ed563f8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:07.362016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7959-12b1-4714-8ff6-9a7e0ed563f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:07.362497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7959-12b1-4714-8ff6-9a7e0ed563f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:07.363354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b7959-12b1-4714-8ff6-9a7e0ed563f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:07.363890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b7959-12b1-4714-8ff6-9a7e0ed563f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:07.365559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b7959-12b1-4714-8ff6-9a7e0ed563f8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:07.380053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 381/759] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:07.388165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:07.426379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14f800-90c5-4555-b888-b9f9e8988913 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:07.426706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14f800-90c5-4555-b888-b9f9e8988913 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:07.426913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14f800-90c5-4555-b888-b9f9e8988913 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:07.428556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 382/760] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:07.433818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17860751-2f39-41fb-9ecd-1f532e325fcd None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:07.434097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17860751-2f39-41fb-9ecd-1f532e325fcd None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:07.434377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17860751-2f39-41fb-9ecd-1f532e325fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:07.459984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-17860751-2f39-41fb-9ecd-1f532e325fcd None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:07.461571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-17860751-2f39-41fb-9ecd-1f532e325fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:07.465710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:07.537278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 383/761] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:07.566726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf75b4-d023-4888-a282-433eed53b3ae None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:07.614358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:07.615269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:07.615722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:07.616138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:07.624959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:07.626565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:07.638503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:07.639660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:07.640810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:07.642972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 379/762] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:26:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:07.650795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17860751-2f39-41fb-9ecd-1f532e325fcd None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:07.678423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:07.794796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17860751-2f39-41fb-9ecd-1f532e325fcd None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:07.828430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:07.829046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:07.829321 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:07.829586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:07.830318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:07.830617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:07.832099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:07.850207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 384/763] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:26:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:07.855804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:07.937407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:08.029087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:08.055684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80b0b-c752-4c85-b619-be7024910ea2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.060488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:08.091359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bdcfd-a6c6-42f7-a208-6a21d9b87001 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.120494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:08.121137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.121413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.121679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.136582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:08.138042 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:08.154110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:08.154677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.154950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.155217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.155831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:08.157221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:08.166473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:08.167409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:08.168330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:08.168330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:08.168330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:08.169010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 380/764] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:08.179378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:08.180243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:08.181232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:08.183231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 385/765] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Aug 22 05:26:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:08.195560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:08.371362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340564-2651-45a7-9397-94efb1855aee None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.380727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7d2f0-45e7-43d4-bbda-1cd5125f2603 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.408068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60c32f-7428-471b-a84f-3779b0973085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:08.408639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60c32f-7428-471b-a84f-3779b0973085 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.411596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60c32f-7428-471b-a84f-3779b0973085 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.411596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60c32f-7428-471b-a84f-3779b0973085 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.411596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60c32f-7428-471b-a84f-3779b0973085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:08.411596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60c32f-7428-471b-a84f-3779b0973085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:08.411963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60c32f-7428-471b-a84f-3779b0973085 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:08.427566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 381/766] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:08.432360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a35d50-89b8-45d1-aadf-123d5bd38925 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.432727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a35d50-89b8-45d1-aadf-123d5bd38925 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.433009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a35d50-89b8-45d1-aadf-123d5bd38925 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.434601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 382/767] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:08.435340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a35d50-89b8-45d1-aadf-123d5bd38925 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:08.443836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff19df-9d05-4e06-a5b7-89d3fb455982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:08.444837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff19df-9d05-4e06-a5b7-89d3fb455982 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.445318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff19df-9d05-4e06-a5b7-89d3fb455982 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.445816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff19df-9d05-4e06-a5b7-89d3fb455982 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.446662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff19df-9d05-4e06-a5b7-89d3fb455982 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:08.447651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff19df-9d05-4e06-a5b7-89d3fb455982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:08.449615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff19df-9d05-4e06-a5b7-89d3fb455982 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:08.452407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:08.457911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 386/768] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 22 05:26:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:08.459177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:08.461005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028770af-ce24-4ffa-bb1b-da69e9ebc723 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.461530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028770af-ce24-4ffa-bb1b-da69e9ebc723 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.462004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028770af-ce24-4ffa-bb1b-da69e9ebc723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.488919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-028770af-ce24-4ffa-bb1b-da69e9ebc723 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:08.490567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-028770af-ce24-4ffa-bb1b-da69e9ebc723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:08.564187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a35d50-89b8-45d1-aadf-123d5bd38925 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.580354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 387/769] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:08.582159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028770af-ce24-4ffa-bb1b-da69e9ebc723 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:08.613040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:08.613683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.613959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.614228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.621895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:08.623488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:08.637244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:08.638169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:08.639063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 383/770] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:08.644643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:08.770515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028770af-ce24-4ffa-bb1b-da69e9ebc723 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.795078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:08.806017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:08.814069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:08.815476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.816251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.816728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.817734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:08.818555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:08.820585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:08.820880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30fbc1-7b26-4add-b7f3-3f05c167bff8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.827389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 388/771] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 22 05:26:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:08.833825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:08.910587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:08.911258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:08.911535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:08.911934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:08.912620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:08.912920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:08.914618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:08.958129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fc2bf-72a6-4148-b43a-c9a219a122ce None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:08.961587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 384/772] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:08.962443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.002868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.003490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.003803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.004070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.004780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] RBAC: Adding query filter params (interface=public, service_id=201538200db04c959a7a89757003f375, region_id=RegionOne) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:09.005085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.006473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.020146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 389/773] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:26:08 2022] GET /identity/v3/endpoints?service_id=201538200db04c959a7a89757003f375&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:09.021089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.086658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:09.111358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d953-e343-4e9d-8402-cbb2cb98968a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:09.132288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeffc2a-ab16-4dfd-be90-8e7e40383190 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:09.158909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.160879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.161169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.161440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] PATH_INFO: `/v3/services/network` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.162034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.163686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.167480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:09.172976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e87cb-1576-4d19-ad55-cd0e8ca1adcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:09.173312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:09.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:09.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 385/774] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:08 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:09.177633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.178480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e87cb-1576-4d19-ad55-cd0e8ca1adcb None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.178891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e87cb-1576-4d19-ad55-cd0e8ca1adcb None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.179171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e87cb-1576-4d19-ad55-cd0e8ca1adcb None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/a3b919bcbb104fa3bf11847b6a35a221/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.181439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e87cb-1576-4d19-ad55-cd0e8ca1adcb None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=a3b919bcbb104fa3bf11847b6a35a221, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.182894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e87cb-1576-4d19-ad55-cd0e8ca1adcb None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.196521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 390/775] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:26:09 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/a3b919bcbb104fa3bf11847b6a35a221/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:26:09.198540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc20200-fe4f-4ba0-8e34-1777d438d521 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.198969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc20200-fe4f-4ba0-8e34-1777d438d521 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.199414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc20200-fe4f-4ba0-8e34-1777d438d521 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.205575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 391/776] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:09.205575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc20200-fe4f-4ba0-8e34-1777d438d521 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.309945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680e80d-ea1f-4d23-9cee-6cd1e9951600 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:09.350800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed1d7a-9479-4372-bd24-8255d4c08392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.351462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed1d7a-9479-4372-bd24-8255d4c08392 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.352462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed1d7a-9479-4372-bd24-8255d4c08392 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.352462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed1d7a-9479-4372-bd24-8255d4c08392 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.352740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed1d7a-9479-4372-bd24-8255d4c08392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.354895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed1d7a-9479-4372-bd24-8255d4c08392 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.361299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc20200-fe4f-4ba0-8e34-1777d438d521 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:09.361752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 386/777] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:09 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:09.362885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52215-6348-4fc8-bcbb-51d4c0834be2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.363469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52215-6348-4fc8-bcbb-51d4c0834be2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.363783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52215-6348-4fc8-bcbb-51d4c0834be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.385298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:09.389595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f1f52215-6348-4fc8-bcbb-51d4c0834be2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:09.391821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f1f52215-6348-4fc8-bcbb-51d4c0834be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:09.412893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.413929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.414759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.415162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.416279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] RBAC: Adding query filter params (name=network) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:09.416692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.418751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.425435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 392/778] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:09.431491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.490771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:09.506887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 387/779] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:09.509139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1e51f-a176-4c8d-b4bb-362c36287c92 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.511938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1e51f-a176-4c8d-b4bb-362c36287c92 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.512229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1e51f-a176-4c8d-b4bb-362c36287c92 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.518075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 388/780] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:09.521105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68d036-3c40-4a03-ab50-0a872def3b61 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.521386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68d036-3c40-4a03-ab50-0a872def3b61 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.521657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68d036-3c40-4a03-ab50-0a872def3b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.543547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6d68d036-3c40-4a03-ab50-0a872def3b61 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:09.544983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6d68d036-3c40-4a03-ab50-0a872def3b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:09.576320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbf255-8e2a-458c-b4b8-7dc755de0535 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:09.612623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.613244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.613520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.613786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.614421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] RBAC: Adding query filter params (type=network) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:09.614721 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.616300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.622658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 393/781] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:09.628093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.635697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 389/782] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:09.684007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:09.738728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd514d72-2c04-47eb-b467-97d67367750a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:09.774655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.775359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.775681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.775955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.776673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] RBAC: Adding query filter params (interface=public, service_id=c1dd60ec52e449a58d9b347efe4dd69a, region_id=RegionOne) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:09.776984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.778370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.784185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 394/783] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:26:09 2022] GET /identity/v3/endpoints?service_id=c1dd60ec52e449a58d9b347efe4dd69a&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:09.789872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68d036-3c40-4a03-ab50-0a872def3b61 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.870242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:09.916227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68d036-3c40-4a03-ab50-0a872def3b61 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:09.937608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:09.950726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:09.951494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:09.951892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:09.952251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:09.953039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:09.954506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:09.960894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 390/784] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:09 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:09.961341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.074907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dac8b4-3e74-42d2-8d38-8b3de438e7c7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:10.112696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:10.113510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:10.113878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:10.114260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:10.125326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ddba1-fe56-476f-b326-9be46f6a59e6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:10.130720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:10.136069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:10.137008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:10.138078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 391/785] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:26:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:10.140980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:10.144520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.171646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:10.172707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:10.173174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:10.173647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:10.185412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:10.191022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:10.191979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:10.192858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:10.195559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 395/786] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:10.202890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.207016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:10.273219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dae20-83ee-47a8-a796-1ad771d9f96d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:10.348997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:10.349626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:10.349961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:10.350235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:10.351213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:10.351611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:10.353000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:10.377894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ea521-efc7-4d21-85bb-4ccd47519d41 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:10.379764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 392/787] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:26:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:10.383362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.444441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:10.445808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:10.446094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:10.446366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:10.447063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:10.447366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:10.448826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:10.463482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 396/788] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:10.468999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.512627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec140bf6-a037-4b3c-8402-9ebcc9e12ab4 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:10.548854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:10.549593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:10.549869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:10.550142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:10.561087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:10.564262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:10.566005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:10.575960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:10.584829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:10.586796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:10.587992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:10.587992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:10.587992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:10.587992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 393/789] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:10.593420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.699707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51113c14-b994-43b1-b0f6-ce9a43d7bd39 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:10.736019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:10.736671 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:10.736950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:10.737225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:10.748076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:10.749558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:10.765104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:10.766277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:10.767445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:10.767445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:10.767445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:10.767445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 397/790] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:10.776627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.784737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ea4c-2610-49a8-a78a-d223d7116944 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:10.839477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:10.840435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:10.840850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:10.841253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:10.842038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:10.842491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:10.844316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:10.871850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 394/791] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:10.878540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:10.928524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:10.947973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:10.970858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb518d8-096a-45cf-bc06-a265039b3d45 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.035326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.035969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.036246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.038033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.038749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:11.039113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.040460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.065484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 398/792] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:11.067539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf8c14-d3a5-4c5b-8a42-5fc5f45c97ba None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.070979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:11.105892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.106538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.106811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.107078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.125461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.126931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:11.151605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:11.153038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:11.154196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 395/793] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:26:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:11.160512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:11.289993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f68a5-7695-4d04-9266-9b1c57cfab03 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.310200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:11.339040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbb8fe-62e5-412e-a8b7-855684295cd2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.346557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.347662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.347938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.348202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.359930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.361401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:11.383958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:11.384959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:11.386542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 399/794] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:11.390759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:11.395885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:11.410910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.411534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.411830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.412096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.412839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:11.413170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.414601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.432475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 396/795] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 22 05:26:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:11.437715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:11.572226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f03724-c560-407c-b9be-8ff67bea57df None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.574069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee85c98-768b-4cac-8197-426c3bc66ce9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.607203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.607979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.608354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.608621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.609387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e52d1-cf7f-42a8-b6d0-64714c39e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.610121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e52d1-cf7f-42a8-b6d0-64714c39e887 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.610549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e52d1-cf7f-42a8-b6d0-64714c39e887 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.610913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e52d1-cf7f-42a8-b6d0-64714c39e887 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.611828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e52d1-cf7f-42a8-b6d0-64714c39e887 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:11.612232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e52d1-cf7f-42a8-b6d0-64714c39e887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.613761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e52d1-cf7f-42a8-b6d0-64714c39e887 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.617435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:11.619414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.621196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.635617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 400/796] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 22 05:26:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:11.637273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ff9e4-072f-497c-a343-0fbacbabbcf6 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.637756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ff9e4-072f-497c-a343-0fbacbabbcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.638225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ff9e4-072f-497c-a343-0fbacbabbcf6 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:11.643263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:11.644409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:11.645529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:11.645529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:11.645529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:11.645529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 397/797] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:11.645529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:11.646667 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 401/798] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:11.648270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e0b05-ac75-43e1-9d45-b669d87b3080 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.648703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e0b05-ac75-43e1-9d45-b669d87b3080 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.649181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e0b05-ac75-43e1-9d45-b669d87b3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.667982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e64e0b05-ac75-43e1-9d45-b669d87b3080 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:11.669502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e64e0b05-ac75-43e1-9d45-b669d87b3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:11.737176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:11.755599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 402/799] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:11.757163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e0b05-ac75-43e1-9d45-b669d87b3080 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:11.760922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749be9-7d7a-4ce1-a1ea-75e32bd37a95 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.805379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf51fd-fb5a-4568-9dff-365f222059a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.806050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf51fd-fb5a-4568-9dff-365f222059a1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.806333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf51fd-fb5a-4568-9dff-365f222059a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.806593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf51fd-fb5a-4568-9dff-365f222059a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.818538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf51fd-fb5a-4568-9dff-365f222059a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.820311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf51fd-fb5a-4568-9dff-365f222059a1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-bbaf51fd-fb5a-4568-9dff-365f222059a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:11.835646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:11.836583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:11.838054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:11.838054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:11.838054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:11.838054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 398/800] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:11.838054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d88dc9-632c-49f8-9513-d3076b0023e4 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.838365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d88dc9-632c-49f8-9513-d3076b0023e4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.838434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d88dc9-632c-49f8-9513-d3076b0023e4 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.843869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 399/801] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:11.844626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d88dc9-632c-49f8-9513-d3076b0023e4 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:11.894806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:11.898118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e0b05-ac75-43e1-9d45-b669d87b3080 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.947384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ece322-40ef-4d08-89e3-a009441f5986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:11.948047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ece322-40ef-4d08-89e3-a009441f5986 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.948324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ece322-40ef-4d08-89e3-a009441f5986 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.948595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ece322-40ef-4d08-89e3-a009441f5986 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.949532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ece322-40ef-4d08-89e3-a009441f5986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:11.949843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ece322-40ef-4d08-89e3-a009441f5986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:11.951242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ece322-40ef-4d08-89e3-a009441f5986 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:11.966460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 403/802] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:11.967990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f87fb8-ba1d-4d50-839f-f6e38fd7ff54 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:11.968280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f87fb8-ba1d-4d50-839f-f6e38fd7ff54 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:11.968531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d88dc9-632c-49f8-9513-d3076b0023e4 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:11.968986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f87fb8-ba1d-4d50-839f-f6e38fd7ff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:11.993820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-78f87fb8-ba1d-4d50-839f-f6e38fd7ff54 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:11.993820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-78f87fb8-ba1d-4d50-839f-f6e38fd7ff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:12.017327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.018334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.019307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.019853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.020974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:12.021726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.024084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.039437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 400/803] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:12.040588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.057303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:12.079079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 404/804] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:12.080310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f87fb8-ba1d-4d50-839f-f6e38fd7ff54 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.162682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79898c-4699-4ba7-aadf-5009856872c1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.166348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:12.192670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f87fb8-ba1d-4d50-839f-f6e38fd7ff54 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.203468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.204169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.204439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.204710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.211909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.213480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:12.225422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:12.226875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 401/805] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:12.228330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.232170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7118cb3-b096-4aff-8139-6cf58728da59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.232822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7118cb3-b096-4aff-8139-6cf58728da59 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.233090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7118cb3-b096-4aff-8139-6cf58728da59 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.233363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7118cb3-b096-4aff-8139-6cf58728da59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.240339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7118cb3-b096-4aff-8139-6cf58728da59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.242074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7118cb3-b096-4aff-8139-6cf58728da59 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-d7118cb3-b096-4aff-8139-6cf58728da59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:12.252819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:12.253679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:12.254560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 405/806] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:12.255562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0154b62-cc74-4899-a73b-bc1b83e3f56a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.255562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0154b62-cc74-4899-a73b-bc1b83e3f56a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.255562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0154b62-cc74-4899-a73b-bc1b83e3f56a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.257374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 406/807] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:12.261795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0154b62-cc74-4899-a73b-bc1b83e3f56a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.311482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:12.338244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0105f-176a-4a3a-a16a-eb01991d091e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.370856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0154b62-cc74-4899-a73b-bc1b83e3f56a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.372688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.373326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.373598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.373864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.384310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:12.390206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:12.391150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 402/808] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:12.392054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.406976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25491ec7-4f57-4cc8-9594-99a32d6921c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.407606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25491ec7-4f57-4cc8-9594-99a32d6921c9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.407893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25491ec7-4f57-4cc8-9594-99a32d6921c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.408194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25491ec7-4f57-4cc8-9594-99a32d6921c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.408850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25491ec7-4f57-4cc8-9594-99a32d6921c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:12.409153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25491ec7-4f57-4cc8-9594-99a32d6921c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.410583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25491ec7-4f57-4cc8-9594-99a32d6921c9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.417214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:12.422492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 407/809] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:12.423840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75664d-0595-4d38-a45e-02e98dad8c40 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.424079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75664d-0595-4d38-a45e-02e98dad8c40 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.424354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75664d-0595-4d38-a45e-02e98dad8c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.443088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6e75664d-0595-4d38-a45e-02e98dad8c40 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:12.444454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6e75664d-0595-4d38-a45e-02e98dad8c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:12.490249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444af21a-94de-49ab-96ff-6b1d00168803 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.512054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 408/810] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:12.512883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75664d-0595-4d38-a45e-02e98dad8c40 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.536967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.537587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.538041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.538326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.539328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:12.539899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.541508 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.541930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:12.555675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 403/811] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:12.556495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.616420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75664d-0595-4d38-a45e-02e98dad8c40 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.642352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:12.649648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.650524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.650745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.650999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.651597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.653001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:12.662006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:12.663011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 05:26:12.664381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:12.665557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966ae4-6cd4-448e-8da8-748f453fbebd None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.669073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 409/812] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:12.669886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.705609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.706305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.706582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.706980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.708211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:12.708768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.710882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.719820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 404/813] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:12.720642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.776573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:12.782720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde45df-14e5-4c76-b7ae-1ecb8a82c424 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.814794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.815453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.815758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.816031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/9de81bd996d4464d8a299228be695e00/roles/be623ba761364dd593a441e1ca9cd988` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.820830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147c713-6a9b-4c57-9a4d-229583aaff34 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:12.830456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=9de81bd996d4464d8a299228be695e00, role_id=be623ba761364dd593a441e1ca9cd988)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.831935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.844676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 410/814] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:26:12 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/9de81bd996d4464d8a299228be695e00/roles/be623ba761364dd593a441e1ca9cd988 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:26:12.845501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.854216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:12.854919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:12.855201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:12.855476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/b63e2db3e07c4471be6928ff31334816/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:12.857756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=b63e2db3e07c4471be6928ff31334816, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:12.859262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:12.876169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 405/815] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:26:12 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/b63e2db3e07c4471be6928ff31334816/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:26:12.877095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:12.895712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:13.034718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91081b-dc5a-4911-b7e6-ec067e804653 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.044120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c9689-71ec-4828-b2cf-fd966bd09a0e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.072401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.073020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.073292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.073556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.074255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:13.074559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.075959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.081512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 411/816] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:13.081845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.086958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.087649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.087931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.088201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.096214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:13.104170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.105895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:13.127625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:13.128661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:13.129550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:13.129550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:13.129550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:13.129717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 406/817] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:13.130633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.195319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:13.226015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0356fe1-9d3f-4288-aa46-464b318c037b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.248370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520e030-7d75-4f9a-9bd9-0a003e2e0ead None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.267563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.268232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.268757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.269035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] PATH_INFO: `/v3/services/network` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.269630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.271794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:13.285393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:13.286444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 05:26:13.287273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:13.291664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 412/818] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:13.292713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.300591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.301241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.301519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.301789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.302430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:13.302729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.305052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.315671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 407/819] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:13.315671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.408408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:13.435934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf2d95-0dcd-4736-9ac7-c1c3bfaabb9d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.447974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6cf63-32bd-4c2a-bc50-b1d5ed38febc None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.472000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.472960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.473237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.473505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.474148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:13.474446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.475874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.479991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:13.493341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.495062 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.495641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.496065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.496863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] RBAC: Adding query filter params (name=network) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:13.497308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.497748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 413/820] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:13.498576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.499639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.511254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 408/821] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:13.512485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.627722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cfe96-ae64-407a-844f-4cff3998e156 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.640533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6bce7-d108-4c35-9fda-d13fbc742c76 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.674080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.674698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.675043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.675284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.675995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.677408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.682931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:13.688533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.689453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.689956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.691357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.705081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.706797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.708257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 414/822] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:13.709451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:13.726171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:13.728393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:13.729361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:13.731724 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 409/823] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:13.743632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.807502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:13.866501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087adee-be27-45dd-afcc-a30c433e39a2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.939265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.940204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.941673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.942127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.944031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:13.945950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:13.947438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:13.948229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066993a-38df-4bdd-8823-90f716fb491e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:13.975726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 410/824] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:13.992932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:13.996841 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:13.998426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:13.998708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:13.998984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:13.999675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] RBAC: Adding query filter params (type=network) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:14.000019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:14.001429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:14.008582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 415/825] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:14.013959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:14.090581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:14.124681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:14.128359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516bad3-3b28-48e2-a739-e4e71841c9a3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:14.140902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c040bfa-db9b-4bcd-bdd3-28826406826b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:14.194931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:14.196312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:14.197677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:14.199000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:14.199830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:14.199988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:14.200332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:14.200934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:14.202436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:14.204952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:14.246571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 416/826] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:14.305158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:14.306662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:14.426313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:14.435798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 411/827] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:14.437179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:14.677626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c0786-e946-4f03-8d7d-b17403d1fc0c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:14.686258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:14.720239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:14.720887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:14.721163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:14.721430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:14.732742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:14.734284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:14.750960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:14.752004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:14.752878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:14.752878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:14.752878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:14.756971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 417/828] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:14.758456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:14.879127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9f731-cb41-47bb-9521-cdb743274186 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:14.916209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb54ec-fdae-4159-9b4c-5c0cab203220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:14.916844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb54ec-fdae-4159-9b4c-5c0cab203220 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:14.917117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb54ec-fdae-4159-9b4c-5c0cab203220 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:14.917386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb54ec-fdae-4159-9b4c-5c0cab203220 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:14.918035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb54ec-fdae-4159-9b4c-5c0cab203220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:14.918337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb54ec-fdae-4159-9b4c-5c0cab203220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:14.919738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb54ec-fdae-4159-9b4c-5c0cab203220 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:14.932476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 412/829] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:14.937828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.017265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72cf5-9572-4d0d-8e98-29d0d5e56a4e None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.017265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72cf5-9572-4d0d-8e98-29d0d5e56a4e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.017265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72cf5-9572-4d0d-8e98-29d0d5e56a4e None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.018142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 413/830] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:15.022158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb51c2-8edb-472d-9c80-cfd8073b6f53 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.022748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb51c2-8edb-472d-9c80-cfd8073b6f53 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.023041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb51c2-8edb-472d-9c80-cfd8073b6f53 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.024618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 414/831] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:15.026573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcc9be-a37b-4f79-8385-6c4bb834ce9c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.026916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcc9be-a37b-4f79-8385-6c4bb834ce9c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.027193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcc9be-a37b-4f79-8385-6c4bb834ce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.045082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd9aae-2f6e-463c-b18f-695137e79db4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.045371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-cddcc9be-a37b-4f79-8385-6c4bb834ce9c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:15.046536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cddcc9be-a37b-4f79-8385-6c4bb834ce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:15.080148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cccbda-ad4f-4cab-bffd-c845ab4ab1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.080786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cccbda-ad4f-4cab-bffd-c845ab4ab1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.081062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cccbda-ad4f-4cab-bffd-c845ab4ab1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.081395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cccbda-ad4f-4cab-bffd-c845ab4ab1e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.088132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cccbda-ad4f-4cab-bffd-c845ab4ab1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:15.089601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cccbda-ad4f-4cab-bffd-c845ab4ab1e9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e4cccbda-ad4f-4cab-bffd-c845ab4ab1e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:15.101272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:15.102283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:15.103142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:15.106964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 418/832] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:15.108399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:15.108510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0aa1-9b3e-4cad-895e-ab65a111a78e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.108686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0aa1-9b3e-4cad-895e-ab65a111a78e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.108993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0aa1-9b3e-4cad-895e-ab65a111a78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.120715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 415/833] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:15.121695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcc9be-a37b-4f79-8385-6c4bb834ce9c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.127522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e24d0aa1-9b3e-4cad-895e-ab65a111a78e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:15.128755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e24d0aa1-9b3e-4cad-895e-ab65a111a78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:15.143013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:15.223906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 419/834] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:15.240323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcc9be-a37b-4f79-8385-6c4bb834ce9c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.275244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.275902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.276173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.276497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.277223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:15.277526 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:15.279023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:15.291288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 416/835] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:15.300331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0aa1-9b3e-4cad-895e-ab65a111a78e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.403694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.406005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0aa1-9b3e-4cad-895e-ab65a111a78e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.449448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.450116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.450350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.450920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.451706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=62d5a029139c4245a36acda923371996, user.id=a3b919bcbb104fa3bf11847b6a35a221) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:15.452014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:15.453895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:15.462041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 420/836] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/role_assignments?user.id=a3b919bcbb104fa3bf11847b6a35a221&scope.project.id=62d5a029139c4245a36acda923371996&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:15.463976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:15.510623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.517324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302ca8d-de6c-4b9c-9f2a-d900f8960689 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.528877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:15.555320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.556171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.556544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.556906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.569140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:15.575091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:15.576340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:15.577278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:15.578194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 417/837] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:15.584182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.631533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c98c8-732f-4ac7-96b5-d1692875cd4a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.672208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.672844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.673116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.673381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.687902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:15.693142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:15.694059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:15.699834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 421/838] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:15.702113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.708128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2956b7-67a8-4352-93cd-96804d2fb23b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.719649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:15.751371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.752301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.753218 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.753633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.754471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:15.755346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:15.757583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:15.769244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 418/839] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:15.774498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.790291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:15.829573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102357a7-3b21-4783-a3f9-e000d89ed86b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.872329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.873636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.874146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.874586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.875549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:15.876282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:15.878210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:15.881688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beda32-cdff-4692-ab86-1f443a099978 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:15.889214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 422/840] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:15.894511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.921756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:15.922875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:15.923332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:15.923889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:15.937422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:15.939448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:15.956334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:15.957563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:15.958668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:15.958668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:15.958668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:15.963922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 419/841] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:15.965309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:15.993230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.013233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed880a-b1e0-4252-95f7-10a0532b4519 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.055095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.055713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.056093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.056425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.058676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.070646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.072595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.079027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603de7d5-3d74-4809-b564-e88e8e237670 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:16.087943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:16.089131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:16.090149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:16.090149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:16.090149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:16.091956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 423/842] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:16.100850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.120011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08769dae-e2de-4c93-90c0-a51c2e070753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.120650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769dae-e2de-4c93-90c0-a51c2e070753 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.120922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769dae-e2de-4c93-90c0-a51c2e070753 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.121185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769dae-e2de-4c93-90c0-a51c2e070753 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.121835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08769dae-e2de-4c93-90c0-a51c2e070753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:16.122138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08769dae-e2de-4c93-90c0-a51c2e070753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.123510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08769dae-e2de-4c93-90c0-a51c2e070753 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.136253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 420/843] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:16.137793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baec1e-6e6f-43a6-8255-2d9de56244cd None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.138079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baec1e-6e6f-43a6-8255-2d9de56244cd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.138345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baec1e-6e6f-43a6-8255-2d9de56244cd None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.144033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 421/844] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:16.144033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baec1e-6e6f-43a6-8255-2d9de56244cd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.218655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f18ed-c38f-4c9c-aadd-8c52d458af22 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.230212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.248382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baec1e-6e6f-43a6-8255-2d9de56244cd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.255871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0c459-6bcb-4a11-afd1-178709d07886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.256477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c459-6bcb-4a11-afd1-178709d07886 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.256824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c459-6bcb-4a11-afd1-178709d07886 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.257210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c459-6bcb-4a11-afd1-178709d07886 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.257867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0c459-6bcb-4a11-afd1-178709d07886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:16.258338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0c459-6bcb-4a11-afd1-178709d07886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.259625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0c459-6bcb-4a11-afd1-178709d07886 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.272756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 424/845] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:16.278625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cfb94-dcd7-4bb5-a4b5-c30e9544d9a2 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.278919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cfb94-dcd7-4bb5-a4b5-c30e9544d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.279192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cfb94-dcd7-4bb5-a4b5-c30e9544d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.285323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.286087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.286372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.286643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.294215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.295691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.297178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8f9cfb94-dcd7-4bb5-a4b5-c30e9544d9a2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:16.298375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8f9cfb94-dcd7-4bb5-a4b5-c30e9544d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:16.307876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:16.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 422/846] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:16.310057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.349098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.379724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 425/847] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:16.381012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cfb94-dcd7-4bb5-a4b5-c30e9544d9a2 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.427687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d2bd2-dbe1-496d-8f02-b1d358130162 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.473123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.473768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.474047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.474321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.485929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cfb94-dcd7-4bb5-a4b5-c30e9544d9a2 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.486427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.487475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:16.502377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:16.503389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:16.509664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 423/848] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:16.510625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.515658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.532330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.532949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.533222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.533579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.534313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:16.534612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.536204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.559940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 426/849] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:16.560802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.596426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.635625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dad486-788d-467b-9a0a-645635d9f0c3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.673071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267298ee-18d1-4bbc-9dac-b9a3d2fd2c47 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.677645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.678453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.679084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.679528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.680425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:16.681304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.683224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.707363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 424/850] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:16.707363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.714402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.715030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.715313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.715600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.772156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.788362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.790059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.813907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fa6cf-3251-48ef-b844-dde18d195fff None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:16.823943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:16.849667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:16.850312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:16.851023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:16.851300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:16.851934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:16.853357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:16.862767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:16.863864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:16.864873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:16.869120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 425/851] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:16.869977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.910920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 427/852] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:16.910920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:16.980827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa6488-a120-4ae3-83c2-84f0432ac1ad None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.006831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.011895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afbb4f-25b2-4618-8040-7e19e6534da1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.018884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.019504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.019814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.020088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.047972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.049429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.049883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.050475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.051594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:17.052322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.054450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.056161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.061057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 428/853] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:17.067326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.094226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.095958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.174024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7087ae8-fff7-403f-abdb-7f5d2259aa72 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.198439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 426/854] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:17.198439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.214082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.214841 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.215177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.215479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.224287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.227711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.229643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:17.245655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:17.246645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:17.247518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:17.247518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:17.247518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:17.251632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 429/855] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:17.252456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.308995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ab84-d019-419b-a599-14bb7d9eb71a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.322110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.346667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.347669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.347947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.348214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.348846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:17.349146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.350746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.355535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 427/856] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:17.356248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.401753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287a464-73b5-4698-b0f3-044a46487726 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.448017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ae1f-72ad-48f0-83c9-870c2d772b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.448678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae1f-72ad-48f0-83c9-870c2d772b88 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.449116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae1f-72ad-48f0-83c9-870c2d772b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.449390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae1f-72ad-48f0-83c9-870c2d772b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.464260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ae1f-72ad-48f0-83c9-870c2d772b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.465755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ae1f-72ad-48f0-83c9-870c2d772b88 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-09c6ae1f-72ad-48f0-83c9-870c2d772b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:17.486516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:17.488019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:17.488957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:17.488957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:17.488957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:17.488957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 430/857] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:17.490274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e739f7-a26f-4b38-8660-1611ef5aee77 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.490274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e739f7-a26f-4b38-8660-1611ef5aee77 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.490274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e739f7-a26f-4b38-8660-1611ef5aee77 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.491207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 431/858] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:17.496132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e739f7-a26f-4b38-8660-1611ef5aee77 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.496583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.507931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019c73b-7ca0-4c20-8d56-0ebe2509608b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.545809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9e2da-7f35-486a-bbfb-d173a684794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.546816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9e2da-7f35-486a-bbfb-d173a684794f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.547093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9e2da-7f35-486a-bbfb-d173a684794f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.547360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9e2da-7f35-486a-bbfb-d173a684794f None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.548366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9e2da-7f35-486a-bbfb-d173a684794f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:17.548472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9e2da-7f35-486a-bbfb-d173a684794f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.549819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9e2da-7f35-486a-bbfb-d173a684794f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.566717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 428/859] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:17.566717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf62a05-f666-4999-a802-9cdffadb4d27 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.566717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf62a05-f666-4999-a802-9cdffadb4d27 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.566717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf62a05-f666-4999-a802-9cdffadb4d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.583383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-dcf62a05-f666-4999-a802-9cdffadb4d27 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:17.584558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-dcf62a05-f666-4999-a802-9cdffadb4d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:17.611966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e739f7-a26f-4b38-8660-1611ef5aee77 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.613500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.649258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.649868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.650142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.650410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.651107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:17.651351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.652769 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.657217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 429/860] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:17.658267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf62a05-f666-4999-a802-9cdffadb4d27 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.670161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 432/861] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:17.675415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.768191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.789277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf62a05-f666-4999-a802-9cdffadb4d27 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.791026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cac0a7-7ee1-4f1e-9199-ec8ec725df8e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:17.826246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.826889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.827167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.827436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.828799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:17.829640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:17.830119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:17.830590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:17.834655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.836286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.838754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:17.840443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:17.848320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:17.849275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:17.850160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 430/862] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:17.855207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:17.856057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:17.856915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:17.857806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:17.860198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 433/863] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:17.871596 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:17.908179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:17.967524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143984f8-ba34-4be2-aba3-1e49a3dc8cd8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:18.001956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a96f3-a7df-46dc-bfe6-2d3aafc5e49c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:18.004304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:18.004922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:18.005204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:18.005482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:18.006181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:18.006487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:18.008230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:18.017956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:18.026541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 431/864] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:18.027408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:18.038035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:18.038660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:18.038938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:18.039508 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:18.040244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:18.040541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:18.041980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:18.054226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 434/865] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:18.060026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:18.129415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53b459-c04e-49c5-b15f-83d067bdec48 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:18.142798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:18.162919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d823b8c-cb33-4963-8dcd-e32aa0226e80 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:18.165277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:18.165915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:18.166197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:18.166466 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:18.167054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:18.168447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:18.177517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:18.178430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:18.179603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:18.183650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 432/866] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Aug 22 05:26:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:18.184510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:18.200330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:18.200966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:18.201206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:18.201475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:18.202234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=62d5a029139c4245a36acda923371996, user.id=b63e2db3e07c4471be6928ff31334816) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:18.202538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:18.204174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:18.210623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 435/867] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:18 2022] GET /identity/v3/role_assignments?user.id=b63e2db3e07c4471be6928ff31334816&scope.project.id=62d5a029139c4245a36acda923371996&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:18.211244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:18.280701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:18.328629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ba14a-9265-4cef-92f6-ce0e3775c740 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:18.345255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5484571-ee7f-4e53-81fb-174912090d72 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:18.370241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:18.371189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:18.371752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:18.372190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:18.373734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=9de81bd996d4464d8a299228be695e00) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:18.374221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:18.375867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:18.387476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eceac8-552a-475d-8247-59488934c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:18.387691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 433/868] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:18 2022] GET /identity/v3/role_assignments?user.id=9de81bd996d4464d8a299228be695e00&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:18.387776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eceac8-552a-475d-8247-59488934c639 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:18.387776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eceac8-552a-475d-8247-59488934c639 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:18.387776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eceac8-552a-475d-8247-59488934c639 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:18.387776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eceac8-552a-475d-8247-59488934c639 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:18.387776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eceac8-552a-475d-8247-59488934c639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:18.387776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eceac8-552a-475d-8247-59488934c639 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:18.393868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 436/869] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:18.399847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:18.521131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b7ee-8864-4a48-9063-731b40d509aa None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:18.550022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:18.558232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fe1db-3509-4478-8e55-1a29a5acbece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:18.559151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fe1db-3509-4478-8e55-1a29a5acbece None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:18.559715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fe1db-3509-4478-8e55-1a29a5acbece None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:18.560242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fe1db-3509-4478-8e55-1a29a5acbece None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:18.560954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fe1db-3509-4478-8e55-1a29a5acbece None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:18.561316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fe1db-3509-4478-8e55-1a29a5acbece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:18.562783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fe1db-3509-4478-8e55-1a29a5acbece None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:18.567758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 434/870] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:19.263565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2039a6d-348f-497e-8f79-ef70a77c9186 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:19.264141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2039a6d-348f-497e-8f79-ef70a77c9186 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:19.264141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2039a6d-348f-497e-8f79-ef70a77c9186 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:19.265620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 437/871] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:19.271011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273554d-9bfe-4190-a2cc-8e4d7bd75bee None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:19.271292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273554d-9bfe-4190-a2cc-8e4d7bd75bee None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:19.271559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273554d-9bfe-4190-a2cc-8e4d7bd75bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:19.291083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e273554d-9bfe-4190-a2cc-8e4d7bd75bee None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:19.292286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e273554d-9bfe-4190-a2cc-8e4d7bd75bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:19.360179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 435/872] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:19.622132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2039a6d-348f-497e-8f79-ef70a77c9186 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:19.635297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:19.729226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2039a6d-348f-497e-8f79-ef70a77c9186 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:19.762582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8f4ae-564f-4c2c-9b13-f9fb370bd202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:19.763197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8f4ae-564f-4c2c-9b13-f9fb370bd202 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:19.763468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8f4ae-564f-4c2c-9b13-f9fb370bd202 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:19.763768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8f4ae-564f-4c2c-9b13-f9fb370bd202 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:19.764424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8f4ae-564f-4c2c-9b13-f9fb370bd202 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:19.765789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8f4ae-564f-4c2c-9b13-f9fb370bd202 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:19.783788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 438/873] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:19 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:20.268696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35f202-e4b5-48be-af1f-ee3180655e8a None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.269366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35f202-e4b5-48be-af1f-ee3180655e8a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.269846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35f202-e4b5-48be-af1f-ee3180655e8a None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.271688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 436/874] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:20.277714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d3a94-0148-4ecc-808d-9204c6cb799f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.278724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d3a94-0148-4ecc-808d-9204c6cb799f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.279237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d3a94-0148-4ecc-808d-9204c6cb799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.298666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7a7d3a94-0148-4ecc-808d-9204c6cb799f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:20.300259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7a7d3a94-0148-4ecc-808d-9204c6cb799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:20.364336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:20.378678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057121cc-e45b-4fcf-8749-9c01b18cdd82 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.379227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057121cc-e45b-4fcf-8749-9c01b18cdd82 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.379227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057121cc-e45b-4fcf-8749-9c01b18cdd82 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.379734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 439/875] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:20.383625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 437/876] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:20.386388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b1b2-4fc3-4cb6-8bf3-9765365965c2 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.386676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b1b2-4fc3-4cb6-8bf3-9765365965c2 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.386951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b1b2-4fc3-4cb6-8bf3-9765365965c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.404605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b5c5b1b2-4fc3-4cb6-8bf3-9765365965c2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:20.405790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b5c5b1b2-4fc3-4cb6-8bf3-9765365965c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:20.479724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 440/877] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:20.519331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0db35-9e3d-4e2e-88b5-d583314c6e71 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.519617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0db35-9e3d-4e2e-88b5-d583314c6e71 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.519834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0db35-9e3d-4e2e-88b5-d583314c6e71 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.521426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 438/878] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:20.526768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e6b14-46aa-4ebb-85dc-36f88d318a2f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.527054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e6b14-46aa-4ebb-85dc-36f88d318a2f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.527326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e6b14-46aa-4ebb-85dc-36f88d318a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.546090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-427e6b14-46aa-4ebb-85dc-36f88d318a2f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:20.547272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-427e6b14-46aa-4ebb-85dc-36f88d318a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:20.614045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 441/879] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:20.658717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0db35-9e3d-4e2e-88b5-d583314c6e71 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:20.737601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e6b14-46aa-4ebb-85dc-36f88d318a2f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:20.765601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0db35-9e3d-4e2e-88b5-d583314c6e71 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:20.793240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:20.808460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:20.809116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.809387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.809662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.810260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:20.811228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:20.812141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:20.823124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:20.824196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:20.825450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 439/880] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:20.830470 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:20.865142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e6b14-46aa-4ebb-85dc-36f88d318a2f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:20.902864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:20.903659 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.904032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.904427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.915652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:20.920610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:20.921555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:20.922398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 442/881] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 22 05:26:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:20.923093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:20.945750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6074f7-bc8e-44b6-954c-1e4476e4abea None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:20.981953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:20.982570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:20.982852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:20.983241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:20.983908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:20.984212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:20.985696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:20.990554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 440/882] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:20.991595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:21.035594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d220b-3f6f-4610-b61d-21b761aad399 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:21.048546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:21.052122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:21.070521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:21.071385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.071685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.071958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.072609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:21.074001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:21.090611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 443/883] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:20 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:21.104693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:21.108361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0895c-aac4-415f-a486-f523c49be7d0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:21.165920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:21.167159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.167515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.167912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.169016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:21.169389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:21.171092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:21.184322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 441/884] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 22 05:26:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:21.189635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:21.241051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e80928-61a8-4139-8229-26da2a437781 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:21.286809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:21.287617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.287883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.288154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.288803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:21.289103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:21.290491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:21.295918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 444/885] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:21.328774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014613d3-cb7c-4b7b-abd4-bc350792b615 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:21.340278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:21.374534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:21.375501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.375991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.376414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.389664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:21.391952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:21.408208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:21.409459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:21.410579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:21.410579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:21.410579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:21.412544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 442/886] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:21.418882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:21.465591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:21.538163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744c528-66d2-4404-823a-7d0c93dfb3aa None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:21.572953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:21.573867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.574289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.574702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.575520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:21.576024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:21.577588 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:21.590624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 445/887] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:21.596704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:21.703986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a4357-1d1c-43b2-9692-f067cc7124b8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:21.740314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7224b-d99f-4931-a2c4-4c56e11aad55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:21.741495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7224b-d99f-4931-a2c4-4c56e11aad55 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.741927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7224b-d99f-4931-a2c4-4c56e11aad55 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.742339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7224b-d99f-4931-a2c4-4c56e11aad55 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.750648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7224b-d99f-4931-a2c4-4c56e11aad55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:21.752323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7224b-d99f-4931-a2c4-4c56e11aad55 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-c1b7224b-d99f-4931-a2c4-4c56e11aad55 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:21.763953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:21.765098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:21.766282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:21.768159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 443/888] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:26:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:21.774255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:21.826802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499df2e5-ffac-410d-b611-e54e289b4cfc None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.827084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499df2e5-ffac-410d-b611-e54e289b4cfc None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.827356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499df2e5-ffac-410d-b611-e54e289b4cfc None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.828991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 444/889] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:21.833991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655d136-f755-4cce-bb37-07d66c7a83f4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.834268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655d136-f755-4cce-bb37-07d66c7a83f4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.834541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655d136-f755-4cce-bb37-07d66c7a83f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.857633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a655d136-f755-4cce-bb37-07d66c7a83f4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:21.859033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a655d136-f755-4cce-bb37-07d66c7a83f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:21.876322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:21.878423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:21.880743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df6049-d847-4275-a04b-8a77ac19705d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:21.924749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:21.925604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:21.926025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:21.926466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:21.927330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:21.927813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:21.929461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:21.947625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 446/891] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 22 05:26:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:21.951711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 445/891] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:21.952851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.067048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bbb88-ac6e-42ae-86c1-3d3508a5761e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.102577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.103528 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.104067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.104560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.131986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:22.173042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.174509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.233476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655d136-f755-4cce-bb37-07d66c7a83f4 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.274163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 447/892] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:21 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:22.285557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.351016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655d136-f755-4cce-bb37-07d66c7a83f4 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.384082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:22.395984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.397508 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.397965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.398409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.399334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.401453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.408723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90697-9bc8-4d50-b436-842722ee9e9a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:22.413976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:22.414969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:22.415862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 446/893] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:22.421318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.438745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:22.456160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.457207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.457691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.458177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.471147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.473690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:22.490437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:22.491979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:22.492907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:22.492907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:22.492907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:22.499564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 448/894] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:22.500668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.530409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86aae8-79eb-4e67-bb21-ed29bd0501a7 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.566299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.567191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.567486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.568011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.568651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:22.569300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.571403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.582193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 447/895] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:22.583080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.618346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed32d27-d732-426f-8e87-25634280ce76 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.630058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:22.658303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.659410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.659922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.660388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.661322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:22.661838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.663694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.681618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:22.682247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 449/896] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:22.684337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.706602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fc802-b7f6-432e-8b9b-b6b33fedb681 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.753572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.754242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.754522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.754789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.755778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:22.756101 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.758214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.767612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 448/897] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:22.770430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.808687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442e6a5-84fb-430d-b92c-85c074d2c087 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.852740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.853894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.854308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.854711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.863543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.865502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.876545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f3d80-4e2f-4f23-a390-7e5c1b0e27e2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:22.879607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:22.880680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:22.884165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:22.886951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 450/898] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:22.888000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.930298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:22.930695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:22.930976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:22.931243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:22.932368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:22.932497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] RBAC: Adding query filter params (interface=public, service_id=15c4579623c94613b44be05b111b1815, region_id=RegionOne) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:22.933473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:22.935611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:22.947653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 449/899] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/endpoints?service_id=15c4579623c94613b44be05b111b1815&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:22.952013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:22.968194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:23.036276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d65be2-b84a-458d-87cf-b42d141a1a3c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:23.074169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30401b-0b93-43d1-a894-bc6762dfaf55 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:23.079766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:23.080586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.081104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.081552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.082810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:23.083257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:23.085277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:23.101779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 451/900] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:23.107918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:23.113483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6570a0-5385-4387-bab9-fe713011bef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:23.114281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570a0-5385-4387-bab9-fe713011bef3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.114697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570a0-5385-4387-bab9-fe713011bef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.115127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570a0-5385-4387-bab9-fe713011bef3 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.116038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6570a0-5385-4387-bab9-fe713011bef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:23.118134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6570a0-5385-4387-bab9-fe713011bef3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:23.126332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 450/901] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:22 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:26:23.133434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1ec76-0f08-4d97-b527-182290d90192 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.133718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1ec76-0f08-4d97-b527-182290d90192 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.133976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1ec76-0f08-4d97-b527-182290d90192 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.135523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 451/902] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:23.141862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40e1c1-4189-4762-99db-192db888e5e6 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.142130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40e1c1-4189-4762-99db-192db888e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.142397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40e1c1-4189-4762-99db-192db888e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.167508 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0f40e1c1-4189-4762-99db-192db888e5e6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:23.169045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0f40e1c1-4189-4762-99db-192db888e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:23.223404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:23.235205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bcbdd-2c41-400e-8ba2-016152467799 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:23.264921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:23.276166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 452/903] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:23.281986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1133428-fce1-4a10-a452-53bb3a30b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:23.282604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1133428-fce1-4a10-a452-53bb3a30b67c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.282878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1133428-fce1-4a10-a452-53bb3a30b67c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.283144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1133428-fce1-4a10-a452-53bb3a30b67c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.283912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1133428-fce1-4a10-a452-53bb3a30b67c None admin] RBAC: Adding query filter params (role.id=2714852226cf4f118efdf02118fd93a2, scope.project.id=62d5a029139c4245a36acda923371996, user.id=9de81bd996d4464d8a299228be695e00) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:23.284215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1133428-fce1-4a10-a452-53bb3a30b67c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:23.285659 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1133428-fce1-4a10-a452-53bb3a30b67c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:23.292234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 452/904] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:23 2022] GET /identity/v3/role_assignments?user.id=9de81bd996d4464d8a299228be695e00&scope.project.id=62d5a029139c4245a36acda923371996&role.id=2714852226cf4f118efdf02118fd93a2 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:23.331516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca15a-ae8a-4613-8934-c305a2cd9e91 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.331824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca15a-ae8a-4613-8934-c305a2cd9e91 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.332106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca15a-ae8a-4613-8934-c305a2cd9e91 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.333811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 453/905] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:23.339805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1997d-709d-4887-82ac-7911b640ba74 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.340114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1997d-709d-4887-82ac-7911b640ba74 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.340359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1997d-709d-4887-82ac-7911b640ba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.360785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-55e1997d-709d-4887-82ac-7911b640ba74 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:23.362014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-55e1997d-709d-4887-82ac-7911b640ba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:23.443767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 453/906] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:23.546213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ca15a-ae8a-4613-8934-c305a2cd9e91 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:23.661026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ca15a-ae8a-4613-8934-c305a2cd9e91 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:23.703714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:23.704749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.705220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.705679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.706453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:23.708068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:23.710569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1997d-709d-4887-82ac-7911b640ba74 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:23.718030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:23.718963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:23.719851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:23.722771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 454/907] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Aug 22 05:26:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:23.728702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:23.802386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:23.815873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:23.836996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1997d-709d-4887-82ac-7911b640ba74 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:23.854678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047f51-3ae8-44c7-9723-c335cdaf194f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:23.883938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:23.884710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.885079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.885437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.886379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:23.887763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:23.901531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:23.902730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:23.903273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:23.903818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:23.904837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:23.905319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:23.907599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:23.915153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 455/909] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:23.915645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 454/909] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:23 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:26:23.921098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:24.039986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38fba0-02e8-4fb6-bc79-f5a3ec48faf3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:24.069799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:24.084433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:24.085336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:24.085830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:24.086243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:24.087067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:24.087540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:24.089079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:24.094698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 456/910] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:24.100625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:24.207940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c93b1-1889-4baf-a649-7f37276c99a4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:24.243132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847338f8-3984-4c02-8fd3-034d40277c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:24.244142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847338f8-3984-4c02-8fd3-034d40277c2c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:24.244565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847338f8-3984-4c02-8fd3-034d40277c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:24.244989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847338f8-3984-4c02-8fd3-034d40277c2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:24.245907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847338f8-3984-4c02-8fd3-034d40277c2c None admin] RBAC: Adding query filter params (interface=public, service_id=b0fcad843c394c58a9a377bef310aaea, region_id=RegionOne) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:24.246386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847338f8-3984-4c02-8fd3-034d40277c2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:24.248089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847338f8-3984-4c02-8fd3-034d40277c2c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:24.253793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 455/911] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:26:24 2022] GET /identity/v3/endpoints?service_id=b0fcad843c394c58a9a377bef310aaea&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:24.260952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:24.376474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ac09b-4187-4d52-b193-ce0fd12b2447 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:24.411671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd462c-28c9-49bb-9bd6-ed5850849b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:24.412624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd462c-28c9-49bb-9bd6-ed5850849b18 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:24.413107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd462c-28c9-49bb-9bd6-ed5850849b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:24.413521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd462c-28c9-49bb-9bd6-ed5850849b18 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:24.414297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd462c-28c9-49bb-9bd6-ed5850849b18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:24.415914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd462c-28c9-49bb-9bd6-ed5850849b18 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:24.423557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 457/912] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:24 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:26:25.162337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07034265-af38-4448-a404-16747092c916 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:25.163695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07034265-af38-4448-a404-16747092c916 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:25.163695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07034265-af38-4448-a404-16747092c916 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:25.164490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 456/913] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:25.169985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce678864-692e-49ef-bfb1-be060e8bb18a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:25.170791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce678864-692e-49ef-bfb1-be060e8bb18a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:25.171278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce678864-692e-49ef-bfb1-be060e8bb18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:25.190517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ce678864-692e-49ef-bfb1-be060e8bb18a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:25.192060 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ce678864-692e-49ef-bfb1-be060e8bb18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:25.243091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:25.251681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cce410-bfbc-4647-b918-28db2b3dc474 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:25.251969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cce410-bfbc-4647-b918-28db2b3dc474 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:25.252244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cce410-bfbc-4647-b918-28db2b3dc474 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:25.254996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 457/914] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:25.259925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455e04-638f-4ef0-86b9-a14800956c53 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:25.260203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455e04-638f-4ef0-86b9-a14800956c53 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:25.260471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08455e04-638f-4ef0-86b9-a14800956c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:25.264290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 458/915] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:25.278942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-08455e04-638f-4ef0-86b9-a14800956c53 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:25.280222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-08455e04-638f-4ef0-86b9-a14800956c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:25.282478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:25.359222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 458/916] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:25.544589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce678864-692e-49ef-bfb1-be060e8bb18a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:25.626850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455e04-638f-4ef0-86b9-a14800956c53 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:25.657194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce678864-692e-49ef-bfb1-be060e8bb18a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:25.696015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:25.696638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:25.696923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:25.697197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:25.697786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:25.699384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:25.708699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:25.710489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:25.711495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 459/917] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:25.718835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:25.747839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08455e04-638f-4ef0-86b9-a14800956c53 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:25.769295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:25.792071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:25.793083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:25.793568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:25.794032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:25.801648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:25.807328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:25.812385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:25.813336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:25.814185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:25.816820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 459/918] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 22 05:26:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:25.823837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:25.841370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad49038-ab54-45ff-926d-2185addea9bd None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:25.879097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:25.880183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:25.880671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:25.881142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:25.882047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:25.882575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:25.884193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:25.894357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 460/919] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:25.895499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:25.966857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b3010-e4ec-4a84-a6ef-bfbe73669ca6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.001412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd1a1d-637e-4f8b-80c5-d713d4d5df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.002403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd1a1d-637e-4f8b-80c5-d713d4d5df3c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.003010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd1a1d-637e-4f8b-80c5-d713d4d5df3c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.003372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd1a1d-637e-4f8b-80c5-d713d4d5df3c None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.004080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd1a1d-637e-4f8b-80c5-d713d4d5df3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:26.004389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd1a1d-637e-4f8b-80c5-d713d4d5df3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.005852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd1a1d-637e-4f8b-80c5-d713d4d5df3c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.009349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935c39c-53de-44ac-8d5a-4061a7d6cb48 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.015781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 460/920] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 22 05:26:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:26.020910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c71672-0f92-4cb6-b26c-7b831f584dd9 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.021196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c71672-0f92-4cb6-b26c-7b831f584dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.021472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c71672-0f92-4cb6-b26c-7b831f584dd9 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.022799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 461/921] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:26.023499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c71672-0f92-4cb6-b26c-7b831f584dd9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.027705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.056211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094d872-df86-4d3c-9fd2-4e21d2b37e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.057088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094d872-df86-4d3c-9fd2-4e21d2b37e44 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.057371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094d872-df86-4d3c-9fd2-4e21d2b37e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.057799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094d872-df86-4d3c-9fd2-4e21d2b37e44 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.058740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094d872-df86-4d3c-9fd2-4e21d2b37e44 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:26.059187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094d872-df86-4d3c-9fd2-4e21d2b37e44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.060766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094d872-df86-4d3c-9fd2-4e21d2b37e44 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.066425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 461/922] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 22 05:26:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:26.067662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcf059-0e6c-439f-82b5-9304a85fa4b5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.067949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcf059-0e6c-439f-82b5-9304a85fa4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.068222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcf059-0e6c-439f-82b5-9304a85fa4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.087855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e6bcf059-0e6c-439f-82b5-9304a85fa4b5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:26.089173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e6bcf059-0e6c-439f-82b5-9304a85fa4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:26.092979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.141719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c71672-0f92-4cb6-b26c-7b831f584dd9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.161638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 462/923] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:26.162268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcf059-0e6c-439f-82b5-9304a85fa4b5 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.185819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.187317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.187776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.188413 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.205882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.208088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:26.228524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:26.229846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:26.231020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:26.231020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:26.231020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:26.233510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 462/924] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:26.238891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.285262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcf059-0e6c-439f-82b5-9304a85fa4b5 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.289565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.324703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8999d-686b-4022-bde2-8387ec269af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.325688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8999d-686b-4022-bde2-8387ec269af4 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.326156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8999d-686b-4022-bde2-8387ec269af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.326627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8999d-686b-4022-bde2-8387ec269af4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.327479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8999d-686b-4022-bde2-8387ec269af4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.329091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8999d-686b-4022-bde2-8387ec269af4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.355436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 463/925] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:26.375234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18a99a-806a-478e-86cf-d44529ade9ad None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.391224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.396647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccfd44-e8b8-421f-be01-3a84b3625f0d None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.397284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccfd44-e8b8-421f-be01-3a84b3625f0d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.397738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccfd44-e8b8-421f-be01-3a84b3625f0d None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.399257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 464/926] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:26.406600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5374f3-f773-4ce4-9747-a7ae26717091 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.406931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5374f3-f773-4ce4-9747-a7ae26717091 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.407174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5374f3-f773-4ce4-9747-a7ae26717091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.425302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.425983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.426321 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.426491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-8a5374f3-f773-4ce4-9747-a7ae26717091 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:26.426596 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.427264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:26.427632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.427743 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8a5374f3-f773-4ce4-9747-a7ae26717091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:26.429487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.444704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 463/927] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:26.451160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.509357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 465/928] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:26.510191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5374f3-f773-4ce4-9747-a7ae26717091 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.577555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c541076-6665-444d-bb70-9eb31a617cae None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.616535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.617658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.618157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.618631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.626742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.628487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.633521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:26.641689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:26.642677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:26.643643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:26.644223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5374f3-f773-4ce4-9747-a7ae26717091 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.647317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 464/929] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:26.657189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.684015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.684656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.684928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.685191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.685774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.687706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.694975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:26.701103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:26.702119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:26.702922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:26.707392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 466/930] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:26.708763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.798973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cff37d-fa7a-4af5-8f93-a5b9da0c63c7 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.828829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7c2a6-b7e0-435d-8133-4f805edf8577 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.836327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.837208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.837786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.838258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.839307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:26.839826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.841555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.864622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:26.864622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:26.864920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:26.865097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:26.865334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 465/931] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:26.866174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:26.866561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.866726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:26.868802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:26.873851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 467/932] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:26.874651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:26.905813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.971419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:26.987623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430edb94-f658-4925-bc88-488cb53cecd5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:26.988810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc04e2-ce3b-4054-9883-d5211c425c87 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.024984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.025748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.026032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.026139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.026332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.026703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.027020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.027129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.027203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.028600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:27.039053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:27.040158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:27.041182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 466/933] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:27.041957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.044251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.045708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:27.060867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:27.061851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:27.062723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:27.062723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:27.062723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:27.062723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 468/934] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:27.062951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.135354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:27.165705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c33c9d-9003-442c-95ca-cb95ebbe2f46 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.168058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9ea10-e323-4279-9036-45bb71fa72d5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.201122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.201849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.202110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.202371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.203068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:27.203452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.203811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.204420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.204695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.204937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.205037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.205575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:27.205874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.207215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.215770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 467/935] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:27.215770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.216883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 469/936] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:27.217598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.233780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:27.326577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a34a8-f6c4-450f-9f38-77448e35e611 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.330097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16273436-8474-49b8-ae8a-6ca862004ed2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.364992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.367293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.367523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.367853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.367958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.368151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.368244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.368470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.368836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] RBAC: Adding query filter params (interface=public, service_id=158b65f4758242d68d9d6dd348d1e7e9, region_id=RegionOne) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:27.369123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:27.369225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.369425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.370535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.370795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.372475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:27.381503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 470/937] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/endpoints?service_id=158b65f4758242d68d9d6dd348d1e7e9&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:27.381503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.387405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 468/938] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:27.388228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.467707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:27.485121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a6f99-e5d5-4027-8d23-340f7139e6aa None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.495326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ca7e-b236-4f5c-8d13-2ab1ffa818d5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.521382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.522029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.522306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.522571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.523196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:27.523494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.524899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.529672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 471/939] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:27.530432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.538643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.539452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.539918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.540348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.541100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.542624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.550703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 469/940] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:26:27.551900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.610071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:27.668293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1def3c-d086-4655-b4fa-c2fb51aef376 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.683215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44355-3f88-487f-b9ed-147d8640e9df None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.712864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.713158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.713434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.713699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.721614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.723122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.734861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.735479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.735808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.736079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.736764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:27.737040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:27.737903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:27.738924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:27.739397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.739479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 472/941] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:27.746466 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:27.752657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:27.775678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 470/942] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:27.872441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e6943-484b-4f78-8274-d864f4987c66 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:27.899964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:27.919605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d06a9-a730-4ea0-828c-a5df7eaaebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:27.920160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d06a9-a730-4ea0-828c-a5df7eaaebba None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:27.920440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d06a9-a730-4ea0-828c-a5df7eaaebba None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:27.920714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d06a9-a730-4ea0-828c-a5df7eaaebba None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:27.921422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d06a9-a730-4ea0-828c-a5df7eaaebba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:27.921727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d06a9-a730-4ea0-828c-a5df7eaaebba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:27.923190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d06a9-a730-4ea0-828c-a5df7eaaebba None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:27.937179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 473/943] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:27.942764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:28.057217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c96e11-d472-4ae3-b014-b44631c2ce98 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:28.091850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def16fb2-9c51-4d88-9628-5db080273f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:28.092521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def16fb2-9c51-4d88-9628-5db080273f62 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:28.092796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def16fb2-9c51-4d88-9628-5db080273f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:28.093070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def16fb2-9c51-4d88-9628-5db080273f62 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/9de81bd996d4464d8a299228be695e00/roles/2714852226cf4f118efdf02118fd93a2` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:28.095360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def16fb2-9c51-4d88-9628-5db080273f62 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=9de81bd996d4464d8a299228be695e00, role_id=2714852226cf4f118efdf02118fd93a2)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:28.096842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def16fb2-9c51-4d88-9628-5db080273f62 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:28.112058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 471/944] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:26:27 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/9de81bd996d4464d8a299228be695e00/roles/2714852226cf4f118efdf02118fd93a2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:26:28.345231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2fd79-ffaa-4e30-a106-9ec8e546f167 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:28.345747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2fd79-ffaa-4e30-a106-9ec8e546f167 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:28.345747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2fd79-ffaa-4e30-a106-9ec8e546f167 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:28.347637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 474/945] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:28.353186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10650-0acb-4570-97bd-63271d47ff30 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:28.353640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10650-0acb-4570-97bd-63271d47ff30 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:28.353817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10650-0acb-4570-97bd-63271d47ff30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:28.372924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-88c10650-0acb-4570-97bd-63271d47ff30 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:28.374140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-88c10650-0acb-4570-97bd-63271d47ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:28.425660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:28.444975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 472/946] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:28.714503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2fd79-ffaa-4e30-a106-9ec8e546f167 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:28.831361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2fd79-ffaa-4e30-a106-9ec8e546f167 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:28.867102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:28.868499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:28.868985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:28.869454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:28.870301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:28.871956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:28.874775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING keystone.server.flask.common [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 05:26:28.893871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.sql.core [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 05:26:28.893871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 05:26:28.893871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [parameters: {'id': 'c2360b1415904adbad26b24355c0e7b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 05:26:28.893871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81584) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2360b1415904adbad26b24355c0e7b9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2360b1415904adbad26b24355c0e7b9. Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application conn.query(q) Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 05:26:28.911701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result.read() Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 22 05:26:28.912539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application next(self.gen) Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application yield resource Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application next(self.gen) Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise self.value Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application session.commit() Aug 22 05:26:28.913551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.session.flush() Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application compat.raise_( Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 05:26:28.914438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 05:26:28.915334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application conn.query(q) Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result.read() Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 05:26:28.916164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [parameters: {'id': 'c2360b1415904adbad26b24355c0e7b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:28.918798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:28.920729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2360b1415904adbad26b24355c0e7b9. Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:28.928710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 475/947] 10.222.0.18 () {60 vars in 1140 bytes} [Mon Aug 22 05:26:28 2022] POST /identity/v3/projects => generated 262 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:28.929457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10650-0acb-4570-97bd-63271d47ff30 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:29.025645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10650-0acb-4570-97bd-63271d47ff30 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:29.061088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847e656-b465-4aa0-abf0-6b07ff47dee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:29.061724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e656-b465-4aa0-abf0-6b07ff47dee0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:29.062000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e656-b465-4aa0-abf0-6b07ff47dee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:29.062263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e656-b465-4aa0-abf0-6b07ff47dee0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:29.069054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847e656-b465-4aa0-abf0-6b07ff47dee0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:29.070502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847e656-b465-4aa0-abf0-6b07ff47dee0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-c847e656-b465-4aa0-abf0-6b07ff47dee0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:29.081371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:29.082254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:29.083144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 473/948] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:29.090847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:29.128728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:29.201862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2875d14-65e7-49a2-806b-44fb548c55e0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:29.237122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea6eaa-6cd0-46bb-9c3d-0585c8766c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:29.237749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea6eaa-6cd0-46bb-9c3d-0585c8766c5e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:29.238028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea6eaa-6cd0-46bb-9c3d-0585c8766c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:29.238300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea6eaa-6cd0-46bb-9c3d-0585c8766c5e None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:29.238979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea6eaa-6cd0-46bb-9c3d-0585c8766c5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:29.239279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea6eaa-6cd0-46bb-9c3d-0585c8766c5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:29.241086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea6eaa-6cd0-46bb-9c3d-0585c8766c5e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:29.263778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 476/949] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:29.634233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41d0bd-aa52-4ce9-b124-ef9c1a81d7df None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:29.634757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41d0bd-aa52-4ce9-b124-ef9c1a81d7df None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:29.634757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41d0bd-aa52-4ce9-b124-ef9c1a81d7df None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:29.636500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 474/950] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:29.642372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98625e18-9055-49e7-9c99-7536a29999b7 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:29.642823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98625e18-9055-49e7-9c99-7536a29999b7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:29.642970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98625e18-9055-49e7-9c99-7536a29999b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:29.661771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-98625e18-9055-49e7-9c99-7536a29999b7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:29.663352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-98625e18-9055-49e7-9c99-7536a29999b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:29.747701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 477/951] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:29.789656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040f17-efea-42b0-9450-a199949985c0 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:29.789978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040f17-efea-42b0-9450-a199949985c0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:29.790215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040f17-efea-42b0-9450-a199949985c0 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:29.791663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 475/952] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:29.797177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2db7-9e8f-4136-b2b5-2579fe56bfba None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:29.797699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2db7-9e8f-4136-b2b5-2579fe56bfba None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:29.798165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2db7-9e8f-4136-b2b5-2579fe56bfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:29.818742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-69cb2db7-9e8f-4136-b2b5-2579fe56bfba None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:29.820743 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-69cb2db7-9e8f-4136-b2b5-2579fe56bfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:29.839172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:29.889125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 478/953] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:30.042318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c040f17-efea-42b0-9450-a199949985c0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.095536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:30.109652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf64d-080f-4977-a88d-23dbfe4c5433 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.109946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf64d-080f-4977-a88d-23dbfe4c5433 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.110206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf64d-080f-4977-a88d-23dbfe4c5433 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.112999 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 479/954] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:30.117872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a7fa4-3d9e-4d39-9150-c4f895c39fef None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.118140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a7fa4-3d9e-4d39-9150-c4f895c39fef None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.118407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a7fa4-3d9e-4d39-9150-c4f895c39fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.136923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0e8a7fa4-3d9e-4d39-9150-c4f895c39fef None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:30.138157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0e8a7fa4-3d9e-4d39-9150-c4f895c39fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:30.160046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c040f17-efea-42b0-9450-a199949985c0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.197178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c803c-9292-4471-9109-240789b9925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.197833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c803c-9292-4471-9109-240789b9925d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.198102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c803c-9292-4471-9109-240789b9925d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.198368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c803c-9292-4471-9109-240789b9925d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.198952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c803c-9292-4471-9109-240789b9925d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.200388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c803c-9292-4471-9109-240789b9925d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.215330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 480/955] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-012c803c-9292-4471-9109-240789b9925d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:30.215521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:30.216657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:30.217586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:30.218428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a7fa4-3d9e-4d39-9150-c4f895c39fef None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.218517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 476/956] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:30.220495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c803c-9292-4471-9109-240789b9925d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.315721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:30.338960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a7fa4-3d9e-4d39-9150-c4f895c39fef None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.342467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c803c-9292-4471-9109-240789b9925d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.356950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:30.377718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.378349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.378624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.378951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.379474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.381046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.390790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:30.390976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:30.392461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:30.393574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 481/957] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:30.395315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.397084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.397084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.397084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:30.397346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.398782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.398890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.408355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 477/958] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:30.412844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.508019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2af037-b51a-42d8-83fe-5812516af483 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.520303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d6ebb-2fc0-4978-bd77-364f077b0d50 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.551094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.551749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.552025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.552299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.552928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:30.553285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.554686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.559108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.564555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 482/959] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:30.564690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.564690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.564690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.565039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.565128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:30.565128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.565128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.569832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 478/960] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:30.574649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.581793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:30.608701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:30.674340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea4ec3-48d3-4a3e-a945-f60389d0a79e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.679085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb69f-ab67-469b-a15f-08d631940a3f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.707252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.707919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.708196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.708457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.717131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.717922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.718328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:30.718522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.718926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.719735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:30.720179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.721726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:30.723902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:30.725048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:30.726195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 483/961] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:30.730545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 479/962] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:30.730683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.731687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.817546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:30.852965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:30.857647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43de97-d7c5-431a-a8cf-46bf8be825cc None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.858690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b6610-fbd2-4d63-847e-21618113a2d9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:30.908499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.908844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:30.909489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.909766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.910040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.910240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:30.910697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:30.910799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:30.911173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.911444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:30.912399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:30.912735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.915014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:30.922153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 480/963] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:30.928627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:30.942490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 484/964] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:31.046392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de553-f829-4076-9625-ea21464943c2 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:31.091472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5fab8-f85f-49b2-b285-faeaa9ffcab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:31.092147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5fab8-f85f-49b2-b285-faeaa9ffcab2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.092419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5fab8-f85f-49b2-b285-faeaa9ffcab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.092683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5fab8-f85f-49b2-b285-faeaa9ffcab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.105089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5fab8-f85f-49b2-b285-faeaa9ffcab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:31.106613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5fab8-f85f-49b2-b285-faeaa9ffcab2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-0cc5fab8-f85f-49b2-b285-faeaa9ffcab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:31.122302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:31.123296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:31.124255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:31.124255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:31.124255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:31.127653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 481/965] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:31.129914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:31.242823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500ce8c-f831-449f-8205-ec808c7466a9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:31.246867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898156f-c6fd-4ebe-b127-c91d600abde8 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.247150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898156f-c6fd-4ebe-b127-c91d600abde8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.247422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898156f-c6fd-4ebe-b127-c91d600abde8 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.248955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 482/966] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:31.254309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de37a-1bf3-4fd7-9bb9-0cb8ef52b480 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.254587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de37a-1bf3-4fd7-9bb9-0cb8ef52b480 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.254853 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de37a-1bf3-4fd7-9bb9-0cb8ef52b480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.274390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b34de37a-1bf3-4fd7-9bb9-0cb8ef52b480 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:31.275608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b34de37a-1bf3-4fd7-9bb9-0cb8ef52b480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:31.276490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:31.285662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:31.286763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.287253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.288025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.288862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:31.289388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:31.291240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:31.304368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 485/967] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:31.305733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:31.312499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:31.357979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 483/968] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:31.415731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258b3cd-4c43-4a7a-8e10-a4d6f83c7a8a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:31.451141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:31.451818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.452095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.452366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.460037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:31.461497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:31.473829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:31.474849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 05:26:31.475903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:31.479692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 486/969] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:26:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:31.481249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de37a-1bf3-4fd7-9bb9-0cb8ef52b480 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:31.579919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de37a-1bf3-4fd7-9bb9-0cb8ef52b480 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:31.618228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:31.619174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.619705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.620194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.621114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:31.621611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:31.623249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:31.630703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:31.639949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:31.643050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 484/970] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 22 05:26:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:31.648897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:31.686474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:31.741517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81673d9-d29e-4b64-8905-fda66c3dd27d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:31.757802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db8080-1417-4a65-80e2-b3b4ed294df0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:31.778035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:31.779056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.779545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.780048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.780877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:31.782772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:31.793048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:31.794028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:31.794139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:31.794313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:31.801689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.sql.core [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 05:26:31.801689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 05:26:31.801689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [parameters: {'user_id': '786f2e721b7d48ec905d11c8d725c904', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 05:26:31.801689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81584) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application conn.query(q) Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result.read() Aug 22 05:26:31.810072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 05:26:31.810984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application next(self.gen) Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application yield resource Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application next(self.gen) Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise self.value Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application session.commit() Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 05:26:31.812008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.session.flush() Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application compat.raise_( Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 05:26:31.812910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 05:26:31.813822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application conn.query(q) Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application result.read() Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 05:26:31.814700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [parameters: {'user_id': '786f2e721b7d48ec905d11c8d725c904', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:31.815543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 22 05:26:31.818398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 22 05:26:31.820798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 22 05:26:31.820798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 22 05:26:31.820798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 05:26:31.820798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 05:26:31.820798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 05:26:31.820798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:31.828227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 487/971] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 22 05:26:31 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:31.829621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:31.864650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:31.868332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:31.870037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:31.950801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdb48a-2199-42ce-92a9-ff682af0c5c2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:31.966261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:32.002667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:32.002667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.002667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.002667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.003556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 485/972] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:31 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:32.004612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:32.012323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:32.014096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:32.035810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:32.036925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 05:26:32.037937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:32.045205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 488/973] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:32.046368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:32.123712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:32.128348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f4c40-c2c1-4e34-8a4c-15519b2b1662 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:32.150460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59607-bc09-41cf-b383-14f40d5d5c68 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:32.168373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:32.169274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.169398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.169653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.180990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:32.182577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:32.186773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:32.187633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.188086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.188528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.189401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:32.189907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:32.191481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:32.198080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:32.199020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:32.199925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:32.199925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:32.199925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:32.203990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 486/974] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:32.205454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:32.239706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 489/975] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 22 05:26:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:32.320801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e829846-e2ef-4406-8c66-53c0318492d7 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:32.361820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0724943-9b34-432d-b407-455031206003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:32.362455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0724943-9b34-432d-b407-455031206003 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.362735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0724943-9b34-432d-b407-455031206003 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.363007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0724943-9b34-432d-b407-455031206003 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.363702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0724943-9b34-432d-b407-455031206003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:32.364010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0724943-9b34-432d-b407-455031206003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:32.365416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0724943-9b34-432d-b407-455031206003 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:32.383805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 487/976] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:32.385650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:32.420557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:32.497170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7596bc-26ed-4dab-997d-ffd52da9b855 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:32.533367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:32.534021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.534300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.534570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.541977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:32.543449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:32.554741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:32.555654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:32.556585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:32.557536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 490/977] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:32.562057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0724943-9b34-432d-b407-455031206003 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:32.574856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:32.666825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0724943-9b34-432d-b407-455031206003 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:32.702610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c332fb5-5248-4a3d-ad37-1a9e698ff727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:32.703231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332fb5-5248-4a3d-ad37-1a9e698ff727 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.703509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332fb5-5248-4a3d-ad37-1a9e698ff727 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.703802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332fb5-5248-4a3d-ad37-1a9e698ff727 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.704498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c332fb5-5248-4a3d-ad37-1a9e698ff727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:32.704800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c332fb5-5248-4a3d-ad37-1a9e698ff727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:32.706244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c332fb5-5248-4a3d-ad37-1a9e698ff727 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:32.717965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 488/978] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:32.724371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:32.772904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a16f9-6630-49a8-897a-6fb16e8b4247 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.773429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a16f9-6630-49a8-897a-6fb16e8b4247 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.773429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a16f9-6630-49a8-897a-6fb16e8b4247 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.774909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 489/979] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:32.779806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5216-83ff-48cf-adda-5c873bf8f1e0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.780085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5216-83ff-48cf-adda-5c873bf8f1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.780355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5216-83ff-48cf-adda-5c873bf8f1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.798724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-edaa5216-83ff-48cf-adda-5c873bf8f1e0 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:32.799958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-edaa5216-83ff-48cf-adda-5c873bf8f1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:32.817832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:32.831247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caca8c4-94ca-42a7-baf7-5003b8b0e45e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:32.868902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf363f8-260b-44da-836a-4679256bc42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:32.869863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf363f8-260b-44da-836a-4679256bc42e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.870294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf363f8-260b-44da-836a-4679256bc42e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.870718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf363f8-260b-44da-836a-4679256bc42e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.871441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf363f8-260b-44da-836a-4679256bc42e None admin] RBAC: Adding query filter params (role.id=2714852226cf4f118efdf02118fd93a2, scope.project.id=62d5a029139c4245a36acda923371996, user.id=9de81bd996d4464d8a299228be695e00) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:32.871776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf363f8-260b-44da-836a-4679256bc42e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:32.873318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf363f8-260b-44da-836a-4679256bc42e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:32.873915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:32.891943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 490/980] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:32.892180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 491/981] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:32 2022] GET /identity/v3/role_assignments?user.id=9de81bd996d4464d8a299228be695e00&scope.project.id=62d5a029139c4245a36acda923371996&role.id=2714852226cf4f118efdf02118fd93a2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:32.984240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9d7c5-248e-4716-9d9d-6f60bebb2cb0 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.984449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9d7c5-248e-4716-9d9d-6f60bebb2cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.984686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9d7c5-248e-4716-9d9d-6f60bebb2cb0 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:32.986035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 491/982] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:32.992069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc203-17aa-4233-bc31-0a6d8688805c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:32.992354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc203-17aa-4233-bc31-0a6d8688805c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:32.992626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc203-17aa-4233-bc31-0a6d8688805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:33.013282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-407bc203-17aa-4233-bc31-0a6d8688805c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:33.014825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-407bc203-17aa-4233-bc31-0a6d8688805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:33.086472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 492/983] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:33.158232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9d7c5-248e-4716-9d9d-6f60bebb2cb0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:33.263606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9d7c5-248e-4716-9d9d-6f60bebb2cb0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:33.298697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:33.299306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:33.299605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:33.299877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:33.300543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:33.301926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:33.318943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 492/984] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:33 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:33.371620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bc203-17aa-4233-bc31-0a6d8688805c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:33.462979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:33.484631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bc203-17aa-4233-bc31-0a6d8688805c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:33.528197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:33.528755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:33.529036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:33.529309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:33.529890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:33.531248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:33.540814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:33.541829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:33.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 493/985] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:33.548135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:33.609406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:33.654048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14b0a-5ca2-49ce-80e8-c4b2a133f54e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:33.687783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:33.688399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:33.688673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:33.688939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:33.689570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:33.689869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:33.691241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:33.696584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 493/986] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:33.702298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:33.808988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4aebf1-7bcd-4866-a29b-a7e4874b7dc4 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:33.843998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:33.844608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:33.844879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:33.845142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:33.845750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:33.846046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:33.847820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:33.853041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 494/987] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:33.858956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:33.969410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084502e4-f49b-40e4-84e3-b78fb639842d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:33.993934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:34.009062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c62d1-198c-46c2-84e2-65e8414856ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:34.009683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c62d1-198c-46c2-84e2-65e8414856ee None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:34.009956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c62d1-198c-46c2-84e2-65e8414856ee None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:34.010220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c62d1-198c-46c2-84e2-65e8414856ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:34.011093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c62d1-198c-46c2-84e2-65e8414856ee None admin] RBAC: Adding query filter params (interface=admin, service_id=158b65f4758242d68d9d6dd348d1e7e9, region_id=RegionOne) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:34.011398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c62d1-198c-46c2-84e2-65e8414856ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:34.012857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c62d1-198c-46c2-84e2-65e8414856ee None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:34.023472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 494/988] 10.222.0.18 () {56 vars in 1249 bytes} [Mon Aug 22 05:26:33 2022] GET /identity/v3/endpoints?service_id=158b65f4758242d68d9d6dd348d1e7e9&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:34.024893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:34.034910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:34.126944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55b6b9-5408-47d1-91b9-6dbb6c13e674 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:34.160682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc1a71-bc54-43b6-a166-fbc0ce9e0621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:34.161288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1a71-bc54-43b6-a166-fbc0ce9e0621 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:34.161563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1a71-bc54-43b6-a166-fbc0ce9e0621 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:34.161826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1a71-bc54-43b6-a166-fbc0ce9e0621 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:34.162476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc1a71-bc54-43b6-a166-fbc0ce9e0621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:34.163866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc1a71-bc54-43b6-a166-fbc0ce9e0621 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:34.170195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 495/989] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:34 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:26:34.824111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d450fb-bd9a-4399-baaa-61cd285c83ed None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:34.824503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d450fb-bd9a-4399-baaa-61cd285c83ed None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:34.824895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d450fb-bd9a-4399-baaa-61cd285c83ed None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:34.826926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 495/990] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:34.832374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2433353-77bf-4f1c-a2da-b2e18d2f6bd3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:34.832842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2433353-77bf-4f1c-a2da-b2e18d2f6bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:34.832940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2433353-77bf-4f1c-a2da-b2e18d2f6bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:34.851900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e2433353-77bf-4f1c-a2da-b2e18d2f6bd3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:34.853078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e2433353-77bf-4f1c-a2da-b2e18d2f6bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:34.924715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:34.931722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 496/991] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:35.191357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d450fb-bd9a-4399-baaa-61cd285c83ed None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:35.266255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329829-dfd1-443f-ae2b-e200bfc91846 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:35.266769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329829-dfd1-443f-ae2b-e200bfc91846 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:35.267057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329829-dfd1-443f-ae2b-e200bfc91846 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:35.269104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 497/992] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:35.274075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4aca8a-5b9d-48af-a80d-b78e12b84c40 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:35.274362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4aca8a-5b9d-48af-a80d-b78e12b84c40 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:35.274636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4aca8a-5b9d-48af-a80d-b78e12b84c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:35.292872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-1d4aca8a-5b9d-48af-a80d-b78e12b84c40 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:35.294068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1d4aca8a-5b9d-48af-a80d-b78e12b84c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:35.310610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d450fb-bd9a-4399-baaa-61cd285c83ed None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:35.347275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:35.347942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:35.348225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:35.348495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] PATH_INFO: `/v3/services/image` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:35.349079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:35.350865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:35.359405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:35.360476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:35.361325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 496/993] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Aug 22 05:26:35 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:35.366402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:35.367661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 498/994] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:35.428171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:35.476898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b0c4-bd8f-4823-8f29-b7993243bc79 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:35.511833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:35.512449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:35.512728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:35.513089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:35.513731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] RBAC: Adding query filter params (name=image) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:35.514031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:35.515491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:35.520521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 497/995] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:35.526396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4aca8a-5b9d-48af-a80d-b78e12b84c40 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:35.628412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:35.629726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4aca8a-5b9d-48af-a80d-b78e12b84c40 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:35.661418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:35.669294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:35.669991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:35.670277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:35.670550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:35.671194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:35.671499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:35.673036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:35.678681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 499/996] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:35.740800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df69c7-a616-4c95-a91f-12c10446a715 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:35.759758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:35.779343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:35.780028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:35.780308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:35.780565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:35.781172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:35.782572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:35.792398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:35.793454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:35.794294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:35.795975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 498/997] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:35.799683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:35.913429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4ec63-24ff-4392-b789-bc2d8dfc78d0 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:35.947564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342376a-af90-4688-8e70-9f63e632565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:35.948217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342376a-af90-4688-8e70-9f63e632565a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:35.948558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342376a-af90-4688-8e70-9f63e632565a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:35.948836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342376a-af90-4688-8e70-9f63e632565a None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:35.949517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342376a-af90-4688-8e70-9f63e632565a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:35.949822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342376a-af90-4688-8e70-9f63e632565a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:35.951246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342376a-af90-4688-8e70-9f63e632565a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:35.960759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 500/998] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:35.962060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:36.048119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411609-2361-4bf1-a719-7a5847663a48 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:36.048324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411609-2361-4bf1-a719-7a5847663a48 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:36.048563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411609-2361-4bf1-a719-7a5847663a48 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:36.051740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 501/999] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:36.055194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac255e-8922-49c5-ab2f-35c9631beb05 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:36.055730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac255e-8922-49c5-ab2f-35c9631beb05 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:36.056219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac255e-8922-49c5-ab2f-35c9631beb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:36.071115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a7ae2-3880-4dc7-899b-89b093cad98a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:36.075953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6dac255e-8922-49c5-ab2f-35c9631beb05 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:36.077367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6dac255e-8922-49c5-ab2f-35c9631beb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:36.107378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:36.108035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:36.108305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:36.108571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:36.109204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:36.109501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:36.110877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:36.115889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 499/1000] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:36.121818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:36.156071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:36.174558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 502/1001] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:36.186157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:36.243722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82517473-d694-4e9a-a279-bd9c9e1cf230 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:36.289028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:36.290133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:36.290514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:36.290793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:36.291520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] RBAC: Adding query filter params (interface=public, service_id=03ce9174262e4b2dbdcc8f23121f26ea, region_id=RegionOne) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:36.291912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:36.293299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:36.299320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 500/1002] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:26:36 2022] GET /identity/v3/endpoints?service_id=03ce9174262e4b2dbdcc8f23121f26ea&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:36.305420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac255e-8922-49c5-ab2f-35c9631beb05 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:36.418887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac255e-8922-49c5-ab2f-35c9631beb05 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:36.462745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:36.463482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:36.463810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:36.464158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:36.464383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:36.464798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:36.466329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:36.476771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 503/1003] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:36 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:26:36.604459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fd6e2-a237-4199-848f-ee30b3b3f2c8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:36.617767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:36.650787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:36.651440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:36.651792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:36.652033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:36.652630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:36.654017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:36.663928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:36.664986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:36.665828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 501/1004] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 22 05:26:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:36.671747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:36.731878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:36.785643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac687-5e0b-430d-b5a7-134216c94d73 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:36.836989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:36.837623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:36.837900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:36.838170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:36.838819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:36.839145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:36.840622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:36.846613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 504/1005] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:36.853131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:36.989133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e7ca7-05d6-46a3-9749-4d515fe6d143 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:37.027755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb280e-2a07-4560-8dc3-3f68210e44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:37.028419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb280e-2a07-4560-8dc3-3f68210e44c8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:37.028710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb280e-2a07-4560-8dc3-3f68210e44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:37.029028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb280e-2a07-4560-8dc3-3f68210e44c8 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:37.029714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb280e-2a07-4560-8dc3-3f68210e44c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:37.030015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb280e-2a07-4560-8dc3-3f68210e44c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:37.031494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb280e-2a07-4560-8dc3-3f68210e44c8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:37.037666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 502/1006] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 22 05:26:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:37.043633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:37.175185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1028c-3f1a-4949-9dbd-ec6f49271e74 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:37.200921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:37.223789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e7ca-1d5d-4980-afdf-028b262f0f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:37.224408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e7ca-1d5d-4980-afdf-028b262f0f61 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:37.224802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e7ca-1d5d-4980-afdf-028b262f0f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:37.225054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e7ca-1d5d-4980-afdf-028b262f0f61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:37.225726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e7ca-1d5d-4980-afdf-028b262f0f61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:37.227141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e7ca-1d5d-4980-afdf-028b262f0f61 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:37.252831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 505/1007] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:37.691264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f3a2c-5012-40c5-ad51-3bd0c6cd2b50 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:37.691842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f3a2c-5012-40c5-ad51-3bd0c6cd2b50 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:37.691842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f3a2c-5012-40c5-ad51-3bd0c6cd2b50 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:37.693383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 503/1008] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:37.699135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054055ad-1909-425f-8234-bfac4ff66fae None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:37.699409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054055ad-1909-425f-8234-bfac4ff66fae None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:37.699731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054055ad-1909-425f-8234-bfac4ff66fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:37.719257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-054055ad-1909-425f-8234-bfac4ff66fae None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:37.720485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-054055ad-1909-425f-8234-bfac4ff66fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:37.797516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 506/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:38.056837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f3a2c-5012-40c5-ad51-3bd0c6cd2b50 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:38.097185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:38.165593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f3a2c-5012-40c5-ad51-3bd0c6cd2b50 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:38.199770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8c8c3-733b-40fe-b2e0-4bf44d72e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:38.200376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8c8c3-733b-40fe-b2e0-4bf44d72e345 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:38.200648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8c8c3-733b-40fe-b2e0-4bf44d72e345 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:38.200909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8c8c3-733b-40fe-b2e0-4bf44d72e345 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:38.201549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8c8c3-733b-40fe-b2e0-4bf44d72e345 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:38.202928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8c8c3-733b-40fe-b2e0-4bf44d72e345 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:38.218847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 504/1010] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:38 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:26:38.469089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698064bd-4b4a-425a-aeae-ae9290c0ca4f None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:38.469635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698064bd-4b4a-425a-aeae-ae9290c0ca4f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:38.469635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698064bd-4b4a-425a-aeae-ae9290c0ca4f None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:38.471142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 507/1011] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:38.476807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdebb9f-cce1-4e0f-a383-9f2c57d5e0ec None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:38.477066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdebb9f-cce1-4e0f-a383-9f2c57d5e0ec None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:38.477357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdebb9f-cce1-4e0f-a383-9f2c57d5e0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:38.495765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7bdebb9f-cce1-4e0f-a383-9f2c57d5e0ec None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:38.496995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7bdebb9f-cce1-4e0f-a383-9f2c57d5e0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:38.572690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 505/1012] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:38 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:38.835967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698064bd-4b4a-425a-aeae-ae9290c0ca4f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:38.945802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698064bd-4b4a-425a-aeae-ae9290c0ca4f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:38.947318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:38.983049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:38.983709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:38.983988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:38.984256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:38.984837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:38.986203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:38.994947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:38.996181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 05:26:38.997039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:38.998271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 508/1013] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:39.002028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdebb9f-cce1-4e0f-a383-9f2c57d5e0ec None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:39.029233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:39.113652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdebb9f-cce1-4e0f-a383-9f2c57d5e0ec None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:39.151113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c362d-da0b-4864-a4e8-295f4bbf5059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:39.151750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c362d-da0b-4864-a4e8-295f4bbf5059 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:39.152028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c362d-da0b-4864-a4e8-295f4bbf5059 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:39.152257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c362d-da0b-4864-a4e8-295f4bbf5059 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:39.152914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c362d-da0b-4864-a4e8-295f4bbf5059 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:39.153211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c362d-da0b-4864-a4e8-295f4bbf5059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:39.154575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c362d-da0b-4864-a4e8-295f4bbf5059 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:39.167002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 506/1014] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:39.168522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:39.176049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079b05-966d-4b11-841f-304a59da6a5e None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:39.176303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079b05-966d-4b11-841f-304a59da6a5e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:39.176618 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079b05-966d-4b11-841f-304a59da6a5e None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:39.178214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 507/1015] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:39.183491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348cc27-f3a7-4b42-b6a8-9da449cd9a13 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:39.183780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348cc27-f3a7-4b42-b6a8-9da449cd9a13 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:39.184079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348cc27-f3a7-4b42-b6a8-9da449cd9a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:39.203043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7348cc27-f3a7-4b42-b6a8-9da449cd9a13 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:39.204318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7348cc27-f3a7-4b42-b6a8-9da449cd9a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:39.279649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 508/1016] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:39 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:39.279875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081db17-7051-4788-b8d9-187b85c69251 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:39.317285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:39.317905 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:39.318179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:39.318565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:39.319962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:39.320272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:39.321697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:39.332314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 509/1017] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 22 05:26:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:39.334175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348cc27-f3a7-4b42-b6a8-9da449cd9a13 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:39.355612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:39.439133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348cc27-f3a7-4b42-b6a8-9da449cd9a13 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:39.475335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:39.476168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:39.476592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:39.477049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:39.477855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:39.479424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:39.506272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 509/1018] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:26:39.551911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:39.588417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:39.670530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa2004-6f68-43bb-a1c5-606d100e4b5e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:39.709774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7410f-f0a5-464b-b955-f931d5e6d804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:39.710336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7410f-f0a5-464b-b955-f931d5e6d804 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:39.710613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7410f-f0a5-464b-b955-f931d5e6d804 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:39.710872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7410f-f0a5-464b-b955-f931d5e6d804 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:39.712002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7410f-f0a5-464b-b955-f931d5e6d804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:39.713484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7410f-f0a5-464b-b955-f931d5e6d804 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:39.715165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 510/1019] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:39.721023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:39.793770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:39.826531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4634437-5a65-4d99-9793-f5a2da810a5f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:39.861327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52611dbf-520a-4d2d-9a55-e0d2e706a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:39.861908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611dbf-520a-4d2d-9a55-e0d2e706a63c None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:39.862186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611dbf-520a-4d2d-9a55-e0d2e706a63c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:39.862478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611dbf-520a-4d2d-9a55-e0d2e706a63c None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:39.863135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52611dbf-520a-4d2d-9a55-e0d2e706a63c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:39.864585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52611dbf-520a-4d2d-9a55-e0d2e706a63c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:39.909701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 510/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:26:39 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:40.160861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e679-1eda-4bd6-85a7-3aa880b7651c None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:40.161116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e679-1eda-4bd6-85a7-3aa880b7651c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:40.161391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094e679-1eda-4bd6-85a7-3aa880b7651c None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:40.162812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 511/1021] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:40.168570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148df07-6fa1-41ca-824a-8d56614c6e61 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:40.169051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148df07-6fa1-41ca-824a-8d56614c6e61 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:40.169149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148df07-6fa1-41ca-824a-8d56614c6e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:40.187647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c148df07-6fa1-41ca-824a-8d56614c6e61 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:40.188855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c148df07-6fa1-41ca-824a-8d56614c6e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:40.265079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 511/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:40.527192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094e679-1eda-4bd6-85a7-3aa880b7651c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:40.625091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:40.633783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094e679-1eda-4bd6-85a7-3aa880b7651c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:40.668434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:40.669066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:40.669335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:40.669598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] PATH_INFO: `/v3/services/image` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:40.670171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:40.671602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:40.680410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:40.681422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:40.684614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 512/1023] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Aug 22 05:26:40 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:40.688223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148df07-6fa1-41ca-824a-8d56614c6e61 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:40.746182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:40.792433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148df07-6fa1-41ca-824a-8d56614c6e61 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:40.826137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:40.826739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:40.827017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:40.827285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:40.827943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] RBAC: Adding query filter params (name=image) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:40.828234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:40.829635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:40.834795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 512/1024] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:40.840543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:40.955146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98b410-69fa-4fd1-bc73-15376ae6741e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:40.990552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:40.991478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:40.991926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:40.992350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:40.993163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:40.993616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:40.995133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:41.004976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 513/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:41.006852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:41.110283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14097d5-7fce-41dd-8b26-880b473f1428 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:41.127851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:41.146072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700572f9-3598-47ce-993f-1353b727725a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:41.147117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700572f9-3598-47ce-993f-1353b727725a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:41.147593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700572f9-3598-47ce-993f-1353b727725a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:41.148034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700572f9-3598-47ce-993f-1353b727725a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:41.148925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700572f9-3598-47ce-993f-1353b727725a None admin] RBAC: Adding query filter params (interface=public, service_id=b8da30069c6a4d5b9cdcbb1823550602, region_id=RegionOne) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:41.149400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700572f9-3598-47ce-993f-1353b727725a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:41.150949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700572f9-3598-47ce-993f-1353b727725a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:41.159728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 513/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:26:41 2022] GET /identity/v3/endpoints?service_id=b8da30069c6a4d5b9cdcbb1823550602&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:41.162573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:41.188464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:41.268448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8881a15-f83a-40fa-948a-8b979b4bfca9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:41.303365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee240cf8-d43c-4a13-85e6-e3aab4d30340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:41.304313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee240cf8-d43c-4a13-85e6-e3aab4d30340 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:41.304761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee240cf8-d43c-4a13-85e6-e3aab4d30340 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:41.305240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee240cf8-d43c-4a13-85e6-e3aab4d30340 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:41.306011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee240cf8-d43c-4a13-85e6-e3aab4d30340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:41.307655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee240cf8-d43c-4a13-85e6-e3aab4d30340 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:41.314437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 514/1027] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:26:41 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:26:41.519999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa631a-8944-44f0-ae0e-4d13e10b4707 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:41.520570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa631a-8944-44f0-ae0e-4d13e10b4707 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:41.520570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa631a-8944-44f0-ae0e-4d13e10b4707 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:41.522194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 514/1028] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:41.527418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e6323-d78e-49ac-ac55-a2f0671ec09b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:41.527738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e6323-d78e-49ac-ac55-a2f0671ec09b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:41.528010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e6323-d78e-49ac-ac55-a2f0671ec09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:41.546226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-406e6323-d78e-49ac-ac55-a2f0671ec09b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:41.547876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-406e6323-d78e-49ac-ac55-a2f0671ec09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:41.618352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 515/1029] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:41.866626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4531775-91ed-4b76-99ad-51f288030952 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:41.866816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4531775-91ed-4b76-99ad-51f288030952 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:41.867108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4531775-91ed-4b76-99ad-51f288030952 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:41.868611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 515/1030] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:41.873804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880894e-1482-4880-8a8c-8ce0327535dc None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:41.874283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880894e-1482-4880-8a8c-8ce0327535dc None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:41.874362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880894e-1482-4880-8a8c-8ce0327535dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:41.878821 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4531775-91ed-4b76-99ad-51f288030952 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:41.893142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6880894e-1482-4880-8a8c-8ce0327535dc None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:41.897430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6880894e-1482-4880-8a8c-8ce0327535dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:41.912028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:41.969735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 516/1031] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:42.003453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4531775-91ed-4b76-99ad-51f288030952 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.048447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.049325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.049748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.050214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.061013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:42.065979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:42.066997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:42.068384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:42.071707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 516/1032] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 22 05:26:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:42.074552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880894e-1482-4880-8a8c-8ce0327535dc None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:42.178954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880894e-1482-4880-8a8c-8ce0327535dc None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.213002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.213947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.214423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.214890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.215802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:42.216304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:42.217913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:42.227170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 517/1033] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 22 05:26:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:42.232864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:42.235723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:42.247528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:42.284758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:42.339913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce419b-3dd4-4936-8c9b-7314611a257d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.345603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca8de4-d557-4283-8712-e942d9fe3b05 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.378631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.379294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.379561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.379868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.380403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.381090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:42.381201 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.381359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.381620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.382515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:42.387633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 518/1034] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:42.389937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:42.392975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:42.395697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:42.413443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:42.414467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:42.415344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:42.415344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:42.415344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:42.415344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 517/1035] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:42.420352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:42.477919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:42.502521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605a713-af70-4c36-966c-2a95b4dd98cf None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.521494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:42.529762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29995b8-c042-47c7-bee0-5c3b30dd3d67 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.535949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.536563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.536837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.537158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.537886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:42.539337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:42.562119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 519/1036] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:26:42 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:42.566837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.567464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.567792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.568061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.568776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:42.569077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:42.570848 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:42.589488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 518/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:42.590790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:42.702742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba1ed4-4049-4d72-9f3f-a57727355199 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.735250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:42.742595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.743254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.743759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.744080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.752283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:42.753735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:42.765895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:42.766864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:42.767840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 520/1038] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:26:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:42.774645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:42.878857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3739d44-2582-4b10-a90a-14b1bcd6f79a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:42.919672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22272736-eaeb-4e9b-a753-3645dbf297b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:42.920303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22272736-eaeb-4e9b-a753-3645dbf297b7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:42.920579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22272736-eaeb-4e9b-a753-3645dbf297b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:42.920844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22272736-eaeb-4e9b-a753-3645dbf297b7 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:42.921617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22272736-eaeb-4e9b-a753-3645dbf297b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:42.921911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22272736-eaeb-4e9b-a753-3645dbf297b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:42.923351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22272736-eaeb-4e9b-a753-3645dbf297b7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:42.950057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 519/1039] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:26:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:42.951304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.052336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c03a-489f-4ad4-bcd8-8db806e6cd4e None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:43.087192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:43.087192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.087192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.087192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.145394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:43.156311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:43.157758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:43.226237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcc19d-5c72-4789-859e-8af26e8cae0b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.226571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcc19d-5c72-4789-859e-8af26e8cae0b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.226790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcc19d-5c72-4789-859e-8af26e8cae0b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.228316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 520/1040] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:43.233307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28ffe8-509a-4ae2-bec7-96e52b5fe72c None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.233586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28ffe8-509a-4ae2-bec7-96e52b5fe72c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.233863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28ffe8-509a-4ae2-bec7-96e52b5fe72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.251824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9e28ffe8-509a-4ae2-bec7-96e52b5fe72c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:43.252998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9e28ffe8-509a-4ae2-bec7-96e52b5fe72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:43.257326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 521/1041] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:42 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:43.261010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:43.262704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.325133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 521/1042] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:43.362613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:43.372912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241675d-c851-4fa9-99d5-8fa097c722ca None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:43.407106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:43.407762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.408043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.408319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.418501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:43.419993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:43.434566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:43.435492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:43.436393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:43.436393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:43.436393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:43.439893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 522/1043] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:43.441429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28ffe8-509a-4ae2-bec7-96e52b5fe72c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.544445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28ffe8-509a-4ae2-bec7-96e52b5fe72c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:43.579033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:43.579603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.579875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.580148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.580798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:43.581100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:43.582480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:43.587881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.594330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 522/1044] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:43.600470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.603443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:43.697708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f132a6-35f9-48db-bd44-4f9ad54725f4 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:43.705104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d7143-8dd8-42b7-96b8-aaff08af895a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:43.736070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:43.736743 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.736989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.737256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] PATH_INFO: `/v3/services/image` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.737844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:43.739222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:43.746936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:43.747887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:43.747997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:43.748932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 05:26:43.749765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:43.750489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.750489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.750620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 523/1045] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:43.755112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.759602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:43.760938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:43.767310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:43.772117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:43.773009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:43.773916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 523/1046] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:43.779061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.841219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:43.863197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0896f-9ea1-4e9c-ac8c-3313674f9963 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:43.885805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3ca09-8e5c-41c4-98b5-737a3d3ecb45 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:43.903263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:43.904183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.904668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.905121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.906166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] RBAC: Adding query filter params (name=image) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:43.906544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:43.907977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:43.913723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 524/1047] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:43.919206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:43.925110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:43.926002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:43.926483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:43.926970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:43.927906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:43.928565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:43.930234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:43.943148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 524/1048] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:43.949323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:44.002329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:44.035385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2867f-b0c8-4679-bc18-81edaf8a0271 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:44.061579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029a69-96f6-4216-bbd1-e343d5114c8d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:44.070464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:44.071137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:44.071418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:44.071753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:44.072388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] RBAC: Adding query filter params (type=image) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:44.072689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:44.074056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:44.078874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 525/1049] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:44.084634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:44.090567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:44.104473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b283be8-e75d-497a-be7a-709ad5d4aa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:44.105020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b283be8-e75d-497a-be7a-709ad5d4aa76 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:44.105284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b283be8-e75d-497a-be7a-709ad5d4aa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:44.105553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b283be8-e75d-497a-be7a-709ad5d4aa76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:44.106290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b283be8-e75d-497a-be7a-709ad5d4aa76 None admin] RBAC: Adding query filter params (role.id=2714852226cf4f118efdf02118fd93a2, scope.project.id=62d5a029139c4245a36acda923371996, user.id=a3b919bcbb104fa3bf11847b6a35a221) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:44.106595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b283be8-e75d-497a-be7a-709ad5d4aa76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:44.108076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b283be8-e75d-497a-be7a-709ad5d4aa76 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:44.123612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 525/1050] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:43 2022] GET /identity/v3/role_assignments?user.id=a3b919bcbb104fa3bf11847b6a35a221&scope.project.id=62d5a029139c4245a36acda923371996&role.id=2714852226cf4f118efdf02118fd93a2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:44.207185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29632b6-6ac9-4225-b381-de758ee029fd None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:44.245090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b1979-2c2a-40d7-abb6-7c513f0bfbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:44.245727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1979-2c2a-40d7-abb6-7c513f0bfbd6 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:44.246007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1979-2c2a-40d7-abb6-7c513f0bfbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:44.246279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1979-2c2a-40d7-abb6-7c513f0bfbd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:44.246956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b1979-2c2a-40d7-abb6-7c513f0bfbd6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:44.248386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b1979-2c2a-40d7-abb6-7c513f0bfbd6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:44.254458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:44.272573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 526/1051] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:26:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:26:44.566014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57393-6dda-4f9d-a54d-3e915d4a35ba None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:44.566557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57393-6dda-4f9d-a54d-3e915d4a35ba None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:44.566557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57393-6dda-4f9d-a54d-3e915d4a35ba None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:44.568119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 526/1052] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:44.573351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb27b-e302-48f3-9f6c-b7050bad317d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:44.573605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb27b-e302-48f3-9f6c-b7050bad317d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:44.573913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb27b-e302-48f3-9f6c-b7050bad317d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:44.592349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-fecfb27b-e302-48f3-9f6c-b7050bad317d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:44.593541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fecfb27b-e302-48f3-9f6c-b7050bad317d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:44.678080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 527/1053] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:44.954712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e57393-6dda-4f9d-a54d-3e915d4a35ba None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:45.062324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e57393-6dda-4f9d-a54d-3e915d4a35ba None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:45.097373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:45.098010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:45.098289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:45.098556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:45.108917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:45.113582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:45.114516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:45.115675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 527/1054] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:45.120827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfb27b-e302-48f3-9f6c-b7050bad317d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:45.225329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfb27b-e302-48f3-9f6c-b7050bad317d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:45.243312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:45.264708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:45.264708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:45.265036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:45.265274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:45.265921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:45.266225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:45.267612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:45.277513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 528/1055] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:45.283772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:45.333236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:45.389437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775c598-2573-4c9d-9657-a372461d319b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:45.425402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:45.426038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:45.426488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:45.426762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] PATH_INFO: `/v3/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:45.441485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] RBAC: Authorizing `identity:get_user(user_id=2db7c84c15dc4c7d9fd6edc9d3ee00b1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:45.442942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:45.445204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 528/1056] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:26:45 2022] GET /identity/v3/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:45.451130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:45.556707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398be54-7af3-4e5b-ac41-7137536a34cf None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:45.594160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:45.594789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:45.595052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:45.595322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:45.645599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:45.667136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:45.668697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:45.766807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 529/1057] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:45.772995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:45.878166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca0899-7e11-454f-853d-ddb7a437d1f6 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:45.880582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:45.923697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629ef5a-82c1-4d9a-a182-474a13c271f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:45.924297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629ef5a-82c1-4d9a-a182-474a13c271f1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:45.924531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629ef5a-82c1-4d9a-a182-474a13c271f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:45.924826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629ef5a-82c1-4d9a-a182-474a13c271f1 None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:45.925928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629ef5a-82c1-4d9a-a182-474a13c271f1 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c0dbf4fd8a4594a59de5812260b3bb)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:45.927378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629ef5a-82c1-4d9a-a182-474a13c271f1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:45.929801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 529/1058] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:26:45 2022] GET /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:45.936627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:46.017573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a121f7-0c2e-451b-9fad-16e09dad7552 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.017934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a121f7-0c2e-451b-9fad-16e09dad7552 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.018131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a121f7-0c2e-451b-9fad-16e09dad7552 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.019504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 530/1059] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:46.023032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:46.025264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411d612-c3f8-4fc7-8982-71d8f3ec9f81 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.025759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411d612-c3f8-4fc7-8982-71d8f3ec9f81 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.026110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411d612-c3f8-4fc7-8982-71d8f3ec9f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.043739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78541f84-0739-495a-9caf-6440faf92440 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:46.045301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d411d612-c3f8-4fc7-8982-71d8f3ec9f81 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:46.046604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d411d612-c3f8-4fc7-8982-71d8f3ec9f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:46.079808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430667d-9e36-4fd6-8235-65fe4eb19a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:46.080426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430667d-9e36-4fd6-8235-65fe4eb19a6c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.080706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430667d-9e36-4fd6-8235-65fe4eb19a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.080962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430667d-9e36-4fd6-8235-65fe4eb19a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.081741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430667d-9e36-4fd6-8235-65fe4eb19a6c None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb, user.id=2db7c84c15dc4c7d9fd6edc9d3ee00b1) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:46.082034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430667d-9e36-4fd6-8235-65fe4eb19a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:46.083591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430667d-9e36-4fd6-8235-65fe4eb19a6c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:46.099643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 530/1060] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:45 2022] GET /identity/v3/role_assignments?user.id=2db7c84c15dc4c7d9fd6edc9d3ee00b1&scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:46.131842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 531/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:46.225417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b125b-be45-4fa3-b860-912c87c46322 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.225417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b125b-be45-4fa3-b860-912c87c46322 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.225417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b125b-be45-4fa3-b860-912c87c46322 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.226278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 531/1062] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:46.231849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db9277-f506-42f6-8f5d-e36d980a14a4 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.232142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db9277-f506-42f6-8f5d-e36d980a14a4 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.232418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db9277-f506-42f6-8f5d-e36d980a14a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.252527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-86db9277-f506-42f6-8f5d-e36d980a14a4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:46.253744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-86db9277-f506-42f6-8f5d-e36d980a14a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:46.327917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 532/1063] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:46.395732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b125b-be45-4fa3-b860-912c87c46322 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:46.509572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b125b-be45-4fa3-b860-912c87c46322 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:46.545824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:46.546530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.546807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.547074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.557917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:46.562794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:46.567813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:46.568741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 532/1064] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 22 05:26:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:46.570875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db9277-f506-42f6-8f5d-e36d980a14a4 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:46.591048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:46.596187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:46.607543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:46.684966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db9277-f506-42f6-8f5d-e36d980a14a4 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:46.698163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a08559-5aac-40c0-8ebc-cca59ba9b2d5 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:46.731373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:46.732123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.732371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.732673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.733333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:46.733726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:46.734161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:46.734776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.735053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.735146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:46.735317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.745583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 533/1065] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 22 05:26:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:46.751067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:46.807661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:46.809152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:46.812770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:46.851326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:46.859874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511193f5-56bc-4ed6-b48b-f7879e34f5ad None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:46.899400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:46.900376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:46.900662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:46.900934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:46.915234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:46.916750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:46.928071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 533/1066] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 05:26:46.929450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:46.936437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:46.937390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:46.938324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:46.938324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:46.938324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:46.938324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 534/1067] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:46.944218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:47.045334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6ebfe-231f-4cff-98bf-20a83d8f99da None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.047674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:47.061977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30812f-f312-4060-ad74-ff9a871b2815 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.083613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.084473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.084818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.085279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.097294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.098773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.100684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.101277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.101554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.101827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.102475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:47.102778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.104185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.116034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 535/1068] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:47.116562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:47.117507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:47.118441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:47.118441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:47.118441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:47.122213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:47.122611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 534/1069] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:47.123519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:47.133194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:47.231712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd3d4b-8826-424e-a38e-f15b095394e0 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.231906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56dfc-0b70-4c9e-8643-b9aac0defca2 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.267376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59597b-b573-479e-a24c-58564243d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.267750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.268361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.268463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59597b-b573-479e-a24c-58564243d12b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.268640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.268891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59597b-b573-479e-a24c-58564243d12b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.268996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.269326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59597b-b573-479e-a24c-58564243d12b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.269597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:47.269919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.271423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.277209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59597b-b573-479e-a24c-58564243d12b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.278800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59597b-b573-479e-a24c-58564243d12b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.284143 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 535/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-da59597b-b573-479e-a24c-58564243d12b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:47.295268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:47.296497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:47.297683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:47.299739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 536/1071] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:26:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:47.305996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:47.326178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:47.421498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a35d99-2280-4063-a552-0567b7741460 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.461232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.461860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.462145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.462410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.463160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:47.463464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.464947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.483918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 536/1072] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:26:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:47.489713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59597b-b573-479e-a24c-58564243d12b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:47.564090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:47.600767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59597b-b573-479e-a24c-58564243d12b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.634820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.635461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.635773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.636046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.653280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.654764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:47.669704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:47.670796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:47.672142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:47.672142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:47.672142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:47.672563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 537/1073] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:47.676749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:47.757148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:47.782852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc4e9-7bb2-48cb-a8eb-be06c031bb34 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.816962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2416e-e982-4674-89c1-27783d655f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.817583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2416e-e982-4674-89c1-27783d655f73 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.817859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2416e-e982-4674-89c1-27783d655f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.818130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2416e-e982-4674-89c1-27783d655f73 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.818777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2416e-e982-4674-89c1-27783d655f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:47.819099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2416e-e982-4674-89c1-27783d655f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.820486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2416e-e982-4674-89c1-27783d655f73 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.832814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 537/1074] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:47.838895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:47.945909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca857-26ad-4c8e-8696-419b7d78bb47 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:47.963103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:47.963341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc44064-4af6-4f4a-b0ec-5d5530103682 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.963676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc44064-4af6-4f4a-b0ec-5d5530103682 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.963955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc44064-4af6-4f4a-b0ec-5d5530103682 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.965345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 538/1075] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:47.971296 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bcab9-a65f-40ed-90e0-60de4b1798fa None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.971689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bcab9-a65f-40ed-90e0-60de4b1798fa None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.972328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bcab9-a65f-40ed-90e0-60de4b1798fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.987704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:47.988610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:47.989020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:47.989423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:47.996486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:47.997233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-541bcab9-a65f-40ed-90e0-60de4b1798fa None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:47.997934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:47.998439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-541bcab9-a65f-40ed-90e0-60de4b1798fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:48.009072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:48.009969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:48.010872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 538/1076] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:48.015896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:48.064681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 539/1077] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:48.119901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03151649-2c50-40db-b7bd-819b8c8e1224 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:48.154486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:48.155099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:48.155373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:48.155670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:48.156363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:48.156666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:48.158120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:48.170360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 539/1078] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:48.176599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bcab9-a65f-40ed-90e0-60de4b1798fa None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:48.241473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:48.280308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bcab9-a65f-40ed-90e0-60de4b1798fa None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:48.314742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:48.315395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:48.315695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:48.315961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/a3b919bcbb104fa3bf11847b6a35a221/roles/2714852226cf4f118efdf02118fd93a2` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:48.318248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=a3b919bcbb104fa3bf11847b6a35a221, role_id=2714852226cf4f118efdf02118fd93a2)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:48.320070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:48.328413 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:48.335591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 540/1079] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:26:48 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/a3b919bcbb104fa3bf11847b6a35a221/roles/2714852226cf4f118efdf02118fd93a2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:26:48.376891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:48.444808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cdc70-e3b6-46f5-91c8-30b32df038ca None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:48.486847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:48.487491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:48.487808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:48.488083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:48.498281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:48.502920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:48.503927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:48.504794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 540/1080] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:48.512382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:48.613974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adac5-3f0c-4fb2-8534-eba5b1d7be46 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:48.637367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:48.649956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:48.650634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:48.650951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:48.651228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:48.651933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:48.652232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:48.653613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:48.662291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 541/1081] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:48.668232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:48.768392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:48.773309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da61d2-d818-4d3a-9a05-95f73c733a4a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:48.808983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:48.809613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:48.809891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:48.810180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] PATH_INFO: `/v3/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:48.811318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] RBAC: Authorizing `identity:get_user(user_id=2db7c84c15dc4c7d9fd6edc9d3ee00b1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:48.812812 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:48.814417 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 541/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:26:48 2022] GET /identity/v3/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:48.820553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:48.924682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90362b7b-ad15-4749-9488-cb45af297260 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:48.960587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30553ffd-66dd-47bc-a69f-27a4e1bd86a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:48.961220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553ffd-66dd-47bc-a69f-27a4e1bd86a9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:48.961498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553ffd-66dd-47bc-a69f-27a4e1bd86a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:48.961761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553ffd-66dd-47bc-a69f-27a4e1bd86a9 None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:48.962901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30553ffd-66dd-47bc-a69f-27a4e1bd86a9 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c0dbf4fd8a4594a59de5812260b3bb)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:48.964388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30553ffd-66dd-47bc-a69f-27a4e1bd86a9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:48.966008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 542/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:26:48 2022] GET /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:48.972421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:49.078207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb341-48ee-4734-b38e-9050715f698c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:49.114017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0436ee-2d9d-4715-a3bb-c7b0c6d1ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:49.114666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0436ee-2d9d-4715-a3bb-c7b0c6d1ed71 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:49.114944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0436ee-2d9d-4715-a3bb-c7b0c6d1ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:49.115709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0436ee-2d9d-4715-a3bb-c7b0c6d1ed71 None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:49.117713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0436ee-2d9d-4715-a3bb-c7b0c6d1ed71 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6c0dbf4fd8a4594a59de5812260b3bb, user_id=2db7c84c15dc4c7d9fd6edc9d3ee00b1, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:49.119164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0436ee-2d9d-4715-a3bb-c7b0c6d1ed71 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:49.133348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 542/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:26:48 2022] PUT /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:26:50.205798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1716b-5bd2-4448-9af6-9585a5403f06 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:50.206331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1716b-5bd2-4448-9af6-9585a5403f06 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:50.206331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1716b-5bd2-4448-9af6-9585a5403f06 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:50.207605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 543/1085] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:50.213106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2bf1c-b0b6-46e4-a33b-448785293f00 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:50.213531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2bf1c-b0b6-46e4-a33b-448785293f00 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:50.213654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2bf1c-b0b6-46e4-a33b-448785293f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:50.233007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-94b2bf1c-b0b6-46e4-a33b-448785293f00 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:50.234218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-94b2bf1c-b0b6-46e4-a33b-448785293f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:50.255147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:50.303226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 543/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:50.563616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1716b-5bd2-4448-9af6-9585a5403f06 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:50.621166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:50.667307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1716b-5bd2-4448-9af6-9585a5403f06 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:50.704959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:50.705591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:50.705870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:50.706122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:50.716883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:50.721539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:50.722887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:50.724336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 544/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 22 05:26:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:50.729561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2bf1c-b0b6-46e4-a33b-448785293f00 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:50.830723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2bf1c-b0b6-46e4-a33b-448785293f00 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:50.866181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc915ad-2e31-4870-a3b6-b10bf61d49e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:50.866912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc915ad-2e31-4870-a3b6-b10bf61d49e6 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:50.867196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc915ad-2e31-4870-a3b6-b10bf61d49e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:50.867462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc915ad-2e31-4870-a3b6-b10bf61d49e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:50.868171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc915ad-2e31-4870-a3b6-b10bf61d49e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:50.868471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc915ad-2e31-4870-a3b6-b10bf61d49e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:50.869872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc915ad-2e31-4870-a3b6-b10bf61d49e6 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:50.878554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 544/1088] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 22 05:26:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:50.884623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:50.995238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b5e3-1521-449f-a04b-28d84cf808dd None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:50.995485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b5e3-1521-449f-a04b-28d84cf808dd None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:50.995794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300b5e3-1521-449f-a04b-28d84cf808dd None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:50.997130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 545/1089] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:26:50.997471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4ac9f-75d2-491a-a990-f764cd275b35 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.002204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224abc0-c1ff-4050-b55f-82b9a1e78351 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.002480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224abc0-c1ff-4050-b55f-82b9a1e78351 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.002749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224abc0-c1ff-4050-b55f-82b9a1e78351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.014017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.022320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e224abc0-c1ff-4050-b55f-82b9a1e78351 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:51.023538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e224abc0-c1ff-4050-b55f-82b9a1e78351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:51.036411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.037335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.037634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.037926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.038226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.048691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:51.050160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:51.064714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:51.065865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:51.067003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:51.067003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:51.067003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:51.067003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 545/1090] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:51.072039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.095152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 546/1091] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:26:51.175518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ebebc-510f-4b37-a6e5-d2dd6b0d6ced None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.211444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.212067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.212333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.212597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.213240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:51.213536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:51.214907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:51.227515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 546/1092] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:51.233192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224abc0-c1ff-4050-b55f-82b9a1e78351 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.337614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224abc0-c1ff-4050-b55f-82b9a1e78351 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.360389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.372451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.373092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.373365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.373622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.381150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:51.382610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:51.390555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:26:51.394341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:51.395267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:51.396497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 547/1093] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:51.401302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.412017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.467390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836aaa32-5ada-42dc-8902-71ced1e71eab None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.502258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe112-002b-48dc-988a-74515ea3e639 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.502588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.503216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.503485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.503791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.513993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:26:51.518682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:51.519702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:26:51.520585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:26:51.523718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 547/1094] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:51.525870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.535798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.536409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.536686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.537103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.538067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:51.538373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:51.539845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:51.558250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 548/1095] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:51.564010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.609107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.623795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622adb-c054-443d-aeb3-3f1235eb7ac3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.627392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.658317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.658932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.659216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.659485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.660454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:51.660454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:51.661920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:51.663326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784467da-be6f-4098-a05b-331412c074d1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.670726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 548/1096] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:51.676378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.701817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.702419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.702701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.702977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.770745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:51.772616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:51.773997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19567d6-6bd4-44a6-a763-46cbfa5ae697 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.808476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.809112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.809365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.809630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] PATH_INFO: `/v3/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.810791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] RBAC: Authorizing `identity:get_user(user_id=2db7c84c15dc4c7d9fd6edc9d3ee00b1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:51.812252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:51.813837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 549/1097] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/users/2db7c84c15dc4c7d9fd6edc9d3ee00b1 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:51.819869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.833524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.837837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:51.869525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 549/1098] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:51.875767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:51.921089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732a879-e16a-495b-818b-95a40cc7ff07 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:51.954496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:51.955075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:51.955381 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:51.955676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:51.973551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab800b-4858-450e-b6c8-47e886c15949 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:52.007588 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:52.008261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:52.008540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:52.008810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:52.019260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.020753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.022725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.024212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:26:52.034795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:52.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:26:52.036817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:26:52.036817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:26:52.036817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:52.036817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 550/1099] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:26:52.042801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:52.048728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:52.062963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:52.132966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 550/1100] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 05:26:52.138993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:52.145786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33983d-ba8a-4293-b30d-c4ab0eab6f9c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:52.179676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:52.180315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:52.180588 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:52.180857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:52.181494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:52.181790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.183169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.194768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 551/1101] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:52.200440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:52.241607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d6d9-a8f0-4a59-9f39-0b14ace53a7f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:52.258346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:52.276650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:52.278676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:52.279529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:52.279840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:52.280108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:52.281315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] RBAC: Authorizing `identity:get_project(project_id=f6c0dbf4fd8a4594a59de5812260b3bb)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.282771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.284455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 551/1102] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:26:52 2022] GET /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:52.290846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:52.300163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9f0e3-2757-4bf9-96ab-f6c513cf340b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:52.337061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:52.337672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:52.337967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:52.338236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:52.344931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.346394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:26:52.358250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:26:52.359338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 552/1103] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:26:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:26:52.366055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:52.389322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17298a1-438c-4b2e-917c-c42ec55d531c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:52.429561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:52.430524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:52.430943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:52.431340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:52.432401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb, user.id=2db7c84c15dc4c7d9fd6edc9d3ee00b1) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:52.432814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.434828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.442175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 552/1104] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:52 2022] GET /identity/v3/role_assignments?user.id=2db7c84c15dc4c7d9fd6edc9d3ee00b1&scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:52.476502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d25d7e-b84c-4ddc-b611-cb4ddbefebf9 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:52.516959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59965e4-72b7-44f7-9488-60d75f5fa7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:52.517568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59965e4-72b7-44f7-9488-60d75f5fa7db None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:52.517846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59965e4-72b7-44f7-9488-60d75f5fa7db None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:52.518118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59965e4-72b7-44f7-9488-60d75f5fa7db None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:52.518832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59965e4-72b7-44f7-9488-60d75f5fa7db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:52.519142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59965e4-72b7-44f7-9488-60d75f5fa7db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.520661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59965e4-72b7-44f7-9488-60d75f5fa7db None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.529098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:52.535149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 553/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:26:52.541914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:52.596615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:52.648143 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cfeec-c929-4b18-9b76-5fad28559799 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:52.681052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93863dcf-ef2e-4642-a708-2bc1e2460655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:52.681650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863dcf-ef2e-4642-a708-2bc1e2460655 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:52.681922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863dcf-ef2e-4642-a708-2bc1e2460655 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:52.682190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863dcf-ef2e-4642-a708-2bc1e2460655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:52.682904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93863dcf-ef2e-4642-a708-2bc1e2460655 None admin] RBAC: Adding query filter params (role.id=2714852226cf4f118efdf02118fd93a2, scope.project.id=62d5a029139c4245a36acda923371996, user.id=a3b919bcbb104fa3bf11847b6a35a221) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:52.683203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93863dcf-ef2e-4642-a708-2bc1e2460655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:52.684668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93863dcf-ef2e-4642-a708-2bc1e2460655 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:52.690519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 553/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:52 2022] GET /identity/v3/role_assignments?user.id=a3b919bcbb104fa3bf11847b6a35a221&scope.project.id=62d5a029139c4245a36acda923371996&role.id=2714852226cf4f118efdf02118fd93a2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:54.378905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac8e26-161a-4d44-b933-c13a9d09c380 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:54.379765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac8e26-161a-4d44-b933-c13a9d09c380 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:54.379956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac8e26-161a-4d44-b933-c13a9d09c380 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:54.382147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 554/1107] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:54.388509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96cd5-4cba-4e54-a110-9b9a46c51f0a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:54.389249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96cd5-4cba-4e54-a110-9b9a46c51f0a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:54.389381 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96cd5-4cba-4e54-a110-9b9a46c51f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:54.416634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4da96cd5-4cba-4e54-a110-9b9a46c51f0a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:54.417809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4da96cd5-4cba-4e54-a110-9b9a46c51f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:54.489924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 554/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:54.762499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac8e26-161a-4d44-b933-c13a9d09c380 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:54.898091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac8e26-161a-4d44-b933-c13a9d09c380 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:54.933711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf982cb-ccd9-4686-afc9-a88b3a61bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:54.934332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf982cb-ccd9-4686-afc9-a88b3a61bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:54.934608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf982cb-ccd9-4686-afc9-a88b3a61bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:54.934877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf982cb-ccd9-4686-afc9-a88b3a61bfb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:54.936049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf982cb-ccd9-4686-afc9-a88b3a61bfb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:54.937503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf982cb-ccd9-4686-afc9-a88b3a61bfb8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:54.939068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 555/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:26:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:54.945772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96cd5-4cba-4e54-a110-9b9a46c51f0a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:54.991858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:55.061966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96cd5-4cba-4e54-a110-9b9a46c51f0a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:55.101444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dba3a0-3ca5-4ffe-8a3e-c782225bb241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:55.102352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dba3a0-3ca5-4ffe-8a3e-c782225bb241 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:55.102633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dba3a0-3ca5-4ffe-8a3e-c782225bb241 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:55.102907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dba3a0-3ca5-4ffe-8a3e-c782225bb241 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:55.103560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dba3a0-3ca5-4ffe-8a3e-c782225bb241 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:55.105056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dba3a0-3ca5-4ffe-8a3e-c782225bb241 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:55.131299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 555/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:26:54 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:26:57.061177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c15d-37b5-4989-a315-15a2def4bf90 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:57.062131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c15d-37b5-4989-a315-15a2def4bf90 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:57.062131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c15d-37b5-4989-a315-15a2def4bf90 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:57.064236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 556/1111] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:26:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:26:57.070670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2975a6b-4aa5-443c-9d5c-da9bd8060a48 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:57.072043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2975a6b-4aa5-443c-9d5c-da9bd8060a48 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:57.072500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2975a6b-4aa5-443c-9d5c-da9bd8060a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:57.095166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c2975a6b-4aa5-443c-9d5c-da9bd8060a48 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:26:57.096392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c2975a6b-4aa5-443c-9d5c-da9bd8060a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:26:57.165037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 556/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:26:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:26:57.431775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850c15d-37b5-4989-a315-15a2def4bf90 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:57.502374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:57.541208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850c15d-37b5-4989-a315-15a2def4bf90 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:57.576326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:57.576946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:57.577219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:57.577486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] PATH_INFO: `/v3/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:57.578958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] RBAC: Authorizing `identity:get_role(role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:57.580367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:57.581671 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 557/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:26:57 2022] GET /identity/v3/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:57.588378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2975a6b-4aa5-443c-9d5c-da9bd8060a48 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:57.626948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:57.704676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2975a6b-4aa5-443c-9d5c-da9bd8060a48 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:57.740641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:57.741275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:57.741551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:57.741816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] PATH_INFO: `/v3/users/414160a890754e4aa1f0897fc38d2210` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:57.754783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=414160a890754e4aa1f0897fc38d2210)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:57.756500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:57.758079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 557/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:26:57 2022] GET /identity/v3/users/414160a890754e4aa1f0897fc38d2210 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:57.764141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:57.874347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad33ff-bbb3-4b65-825b-1d84cf24288b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:57.909778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:57.910393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:57.910664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:57.910928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:57.913733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:57.980307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:57.981687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:58.081240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 558/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:26:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:26:58.087865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:58.173408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:58.189383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061823-dc74-408f-99d9-4139a2aa3ec1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:58.224253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b154fac-4699-44e3-a5da-28a1c9d73178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:58.224875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b154fac-4699-44e3-a5da-28a1c9d73178 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:58.225147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b154fac-4699-44e3-a5da-28a1c9d73178 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:58.225409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b154fac-4699-44e3-a5da-28a1c9d73178 None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:58.226552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b154fac-4699-44e3-a5da-28a1c9d73178 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c0dbf4fd8a4594a59de5812260b3bb)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:58.228015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b154fac-4699-44e3-a5da-28a1c9d73178 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:58.229626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 558/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:26:58 2022] GET /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:26:58.235970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:26:58.269884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:26:58.336942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bb0d0-7529-4bb2-b876-5018580f3498 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:26:58.372442 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089e011-38a3-4691-886d-e02ab4c919cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:26:58.373006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089e011-38a3-4691-886d-e02ab4c919cc None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:26:58.373307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089e011-38a3-4691-886d-e02ab4c919cc None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:26:58.373570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089e011-38a3-4691-886d-e02ab4c919cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:26:58.374288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089e011-38a3-4691-886d-e02ab4c919cc None admin] RBAC: Adding query filter params (role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb, user.id=414160a890754e4aa1f0897fc38d2210) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:26:58.374595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089e011-38a3-4691-886d-e02ab4c919cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:26:58.376064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089e011-38a3-4691-886d-e02ab4c919cc None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:26:58.381764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 559/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:26:58 2022] GET /identity/v3/role_assignments?user.id=414160a890754e4aa1f0897fc38d2210&scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:00.239121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074b692-7c93-4b1b-a652-3a9ecb90443c None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:00.239959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074b692-7c93-4b1b-a652-3a9ecb90443c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:00.239959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074b692-7c93-4b1b-a652-3a9ecb90443c None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:00.241972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 559/1118] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:00.251227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858877-7582-4ee6-b71c-f55ecec84e23 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:00.255739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858877-7582-4ee6-b71c-f55ecec84e23 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:00.256019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858877-7582-4ee6-b71c-f55ecec84e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:00.277001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-12858877-7582-4ee6-b71c-f55ecec84e23 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:00.278222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-12858877-7582-4ee6-b71c-f55ecec84e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:00.359115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 560/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:00.624946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074b692-7c93-4b1b-a652-3a9ecb90443c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:00.752322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074b692-7c93-4b1b-a652-3a9ecb90443c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:00.786908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:00.787516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:00.787819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:00.788088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] PATH_INFO: `/v3/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:00.789501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] RBAC: Authorizing `identity:get_role(role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:00.790860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:00.792212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 560/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:00 2022] GET /identity/v3/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:00.798462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858877-7582-4ee6-b71c-f55ecec84e23 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:00.826520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:00.910364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858877-7582-4ee6-b71c-f55ecec84e23 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:00.954170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:00.955182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:00.955182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:00.955899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] PATH_INFO: `/v3/users/414160a890754e4aa1f0897fc38d2210` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:00.956830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] RBAC: Authorizing `identity:get_user(user_id=414160a890754e4aa1f0897fc38d2210)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:00.957917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:00.959792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 561/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:00 2022] GET /identity/v3/users/414160a890754e4aa1f0897fc38d2210 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:00.966487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:00.997255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:01.086818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23b8b1-c9d6-447c-bfa6-6b7df7b899ea None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:01.121444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e90725-1739-454c-85b8-3c5b3416ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:01.122059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90725-1739-454c-85b8-3c5b3416ac49 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:01.122333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90725-1739-454c-85b8-3c5b3416ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:01.122593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e90725-1739-454c-85b8-3c5b3416ac49 None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:01.123759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e90725-1739-454c-85b8-3c5b3416ac49 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c0dbf4fd8a4594a59de5812260b3bb)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:01.125198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e90725-1739-454c-85b8-3c5b3416ac49 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:01.126812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 561/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:00 2022] GET /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:01.133120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:01.229344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:01.236280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1588b0-a254-4b22-b6d7-b09532ea4f92 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:01.273976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54a1ad-4b90-46b6-bebe-0dd578f10dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:01.274663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a1ad-4b90-46b6-bebe-0dd578f10dae None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:01.274883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a1ad-4b90-46b6-bebe-0dd578f10dae None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:01.275150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a1ad-4b90-46b6-bebe-0dd578f10dae None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb/users/414160a890754e4aa1f0897fc38d2210/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:01.277564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54a1ad-4b90-46b6-bebe-0dd578f10dae None admin] RBAC: Authorizing `identity:create_grant(project_id=f6c0dbf4fd8a4594a59de5812260b3bb, user_id=414160a890754e4aa1f0897fc38d2210, role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:01.279011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54a1ad-4b90-46b6-bebe-0dd578f10dae None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:01.295564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 562/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:27:01 2022] PUT /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb/users/414160a890754e4aa1f0897fc38d2210/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:27:03.149765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278ced6-678a-49dc-b34b-dff1e646cbac None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:03.150294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278ced6-678a-49dc-b34b-dff1e646cbac None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:03.150294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278ced6-678a-49dc-b34b-dff1e646cbac None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:03.151647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 562/1124] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:03.157115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b32185-94cd-4318-af96-1e85ae8d990d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:03.157552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b32185-94cd-4318-af96-1e85ae8d990d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:03.157644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b32185-94cd-4318-af96-1e85ae8d990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:03.177678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-98b32185-94cd-4318-af96-1e85ae8d990d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:03.178850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-98b32185-94cd-4318-af96-1e85ae8d990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:03.251807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 563/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:03.520210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278ced6-678a-49dc-b34b-dff1e646cbac None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:03.628142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:03.640192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278ced6-678a-49dc-b34b-dff1e646cbac None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:03.674249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:03.674863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:03.675134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:03.675398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] PATH_INFO: `/v3/roles/3c82fb4e4e0148ffb6a0c1286f1900b3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:03.676873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] RBAC: Authorizing `identity:get_role(role_id=3c82fb4e4e0148ffb6a0c1286f1900b3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:03.678574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:03.679932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 563/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:03 2022] GET /identity/v3/roles/3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:03.686053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b32185-94cd-4318-af96-1e85ae8d990d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:03.777924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:03.793897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b32185-94cd-4318-af96-1e85ae8d990d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:03.828359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:03.828978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:03.829236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:03.829515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] PATH_INFO: `/v3/users/414160a890754e4aa1f0897fc38d2210` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:03.830672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] RBAC: Authorizing `identity:get_user(user_id=414160a890754e4aa1f0897fc38d2210)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:03.832143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:03.833779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 564/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:03 2022] GET /identity/v3/users/414160a890754e4aa1f0897fc38d2210 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:03.843150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:03.970471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019c7c9-95ee-45c7-9010-af662985683a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:04.006275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:04.006884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:04.007154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:04.007416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:04.038386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:04.080804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:04.082250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:04.192725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 564/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:27:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:27:04.199018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:04.305216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d23783-327d-4855-8f59-66100ec245b4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:04.340051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314bece-a57c-4a5d-823c-af84dd55ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:04.340668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314bece-a57c-4a5d-823c-af84dd55ee73 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:04.340944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314bece-a57c-4a5d-823c-af84dd55ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:04.341213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314bece-a57c-4a5d-823c-af84dd55ee73 None admin] PATH_INFO: `/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:04.342365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314bece-a57c-4a5d-823c-af84dd55ee73 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c0dbf4fd8a4594a59de5812260b3bb)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:04.343843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314bece-a57c-4a5d-823c-af84dd55ee73 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:04.345445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 565/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:04 2022] GET /identity/v3/projects/f6c0dbf4fd8a4594a59de5812260b3bb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:04.352130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:04.413136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:04.459974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a116cdb-fad1-4565-a576-4b0358ad2523 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:04.495926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891fc71-4de3-4522-9ea8-d98cc1e7fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:04.496523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891fc71-4de3-4522-9ea8-d98cc1e7fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:04.496795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891fc71-4de3-4522-9ea8-d98cc1e7fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:04.497030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891fc71-4de3-4522-9ea8-d98cc1e7fdb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:04.497769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891fc71-4de3-4522-9ea8-d98cc1e7fdb5 None admin] RBAC: Adding query filter params (role.id=3c82fb4e4e0148ffb6a0c1286f1900b3, scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb, user.id=414160a890754e4aa1f0897fc38d2210) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:04.498066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891fc71-4de3-4522-9ea8-d98cc1e7fdb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:04.499481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891fc71-4de3-4522-9ea8-d98cc1e7fdb5 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:04.505296 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 565/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:27:04 2022] GET /identity/v3/role_assignments?user.id=414160a890754e4aa1f0897fc38d2210&scope.project.id=f6c0dbf4fd8a4594a59de5812260b3bb&role.id=3c82fb4e4e0148ffb6a0c1286f1900b3 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:06.426573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200fa9b-3f22-4897-9059-b2a2d269f792 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:06.427507 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200fa9b-3f22-4897-9059-b2a2d269f792 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:06.427507 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200fa9b-3f22-4897-9059-b2a2d269f792 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:06.429558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 566/1131] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:06.438187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5f8ba-33d9-4a7c-a9f3-da2d3ace13df None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:06.438603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5f8ba-33d9-4a7c-a9f3-da2d3ace13df None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:06.438761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5f8ba-33d9-4a7c-a9f3-da2d3ace13df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:06.459419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c9a5f8ba-33d9-4a7c-a9f3-da2d3ace13df None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:06.460632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c9a5f8ba-33d9-4a7c-a9f3-da2d3ace13df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:06.535960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 566/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:06.818962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200fa9b-3f22-4897-9059-b2a2d269f792 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:06.850431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:06.935756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200fa9b-3f22-4897-9059-b2a2d269f792 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:06.970820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0691b-65e2-4887-8b29-b040e56d3e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:06.971440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0691b-65e2-4887-8b29-b040e56d3e9d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:06.972123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0691b-65e2-4887-8b29-b040e56d3e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:06.972393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0691b-65e2-4887-8b29-b040e56d3e9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:06.973630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0691b-65e2-4887-8b29-b040e56d3e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:06.975453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0691b-65e2-4887-8b29-b040e56d3e9d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:06.977238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 567/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:27:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:06.983621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5f8ba-33d9-4a7c-a9f3-da2d3ace13df None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:07.040153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:07.092685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5f8ba-33d9-4a7c-a9f3-da2d3ace13df None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:07.128018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25eed4d-edc5-4abb-b610-97acf36e3c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:07.128638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25eed4d-edc5-4abb-b610-97acf36e3c56 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:07.128920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25eed4d-edc5-4abb-b610-97acf36e3c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:07.129196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25eed4d-edc5-4abb-b610-97acf36e3c56 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:07.129853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25eed4d-edc5-4abb-b610-97acf36e3c56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:07.131342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25eed4d-edc5-4abb-b610-97acf36e3c56 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:07.160462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 567/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:27:06 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:27:09.093911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c96f0c-5876-400a-95e3-af89e3b24662 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:09.094469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c96f0c-5876-400a-95e3-af89e3b24662 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:09.094469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c96f0c-5876-400a-95e3-af89e3b24662 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:09.095826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 568/1135] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:09.102002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1272803-be9f-46e9-a04b-25d4404fff38 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:09.102422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1272803-be9f-46e9-a04b-25d4404fff38 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:09.102528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1272803-be9f-46e9-a04b-25d4404fff38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:09.125021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e1272803-be9f-46e9-a04b-25d4404fff38 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:09.126207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e1272803-be9f-46e9-a04b-25d4404fff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:09.209757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 568/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:27:09.489889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c96f0c-5876-400a-95e3-af89e3b24662 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:09.613928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:09.625403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c96f0c-5876-400a-95e3-af89e3b24662 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:09.659465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc51b6-525d-42e3-b7ea-7bb27660111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:09.660108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc51b6-525d-42e3-b7ea-7bb27660111e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:09.660383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc51b6-525d-42e3-b7ea-7bb27660111e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:09.660646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc51b6-525d-42e3-b7ea-7bb27660111e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:09.661766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc51b6-525d-42e3-b7ea-7bb27660111e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:09.663209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc51b6-525d-42e3-b7ea-7bb27660111e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:09.664770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 569/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:27:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:09.671399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1272803-be9f-46e9-a04b-25d4404fff38 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:09.711065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:09.778501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1272803-be9f-46e9-a04b-25d4404fff38 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:09.812587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85dd61-d47b-43a5-91bc-5241b44fbe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:09.813183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85dd61-d47b-43a5-91bc-5241b44fbe94 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:09.813453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85dd61-d47b-43a5-91bc-5241b44fbe94 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:09.813713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85dd61-d47b-43a5-91bc-5241b44fbe94 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:09.814344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85dd61-d47b-43a5-91bc-5241b44fbe94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:09.815786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85dd61-d47b-43a5-91bc-5241b44fbe94 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:09.845598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 569/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:27:09 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:27:11.785727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987baa1-e35f-43b1-ab04-cdd5daee6da4 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:11.786418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987baa1-e35f-43b1-ab04-cdd5daee6da4 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:11.786418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987baa1-e35f-43b1-ab04-cdd5daee6da4 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:11.787681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 570/1139] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:11.797499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf816-6ae6-43a9-ab11-fb8d429ee63e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:11.798199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf816-6ae6-43a9-ab11-fb8d429ee63e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:11.798794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf816-6ae6-43a9-ab11-fb8d429ee63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:11.823631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e4dbf816-6ae6-43a9-ab11-fb8d429ee63e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:11.824841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e4dbf816-6ae6-43a9-ab11-fb8d429ee63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:11.907707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 570/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:12.302570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987baa1-e35f-43b1-ab04-cdd5daee6da4 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:12.438876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987baa1-e35f-43b1-ab04-cdd5daee6da4 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:12.472951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:12.473582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:12.474398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:12.474670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:12.484984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:27:12.490251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:27:12.491401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:27:12.492389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 571/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:27:12.498093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf816-6ae6-43a9-ab11-fb8d429ee63e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:12.523160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:12.611980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf816-6ae6-43a9-ab11-fb8d429ee63e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:12.646901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:12.647506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:12.647814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:12.648082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:12.648747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:12.649045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:12.650408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:12.658522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 571/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:27:12.667895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:12.691811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:12.778412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2729d-3020-4572-85c2-bc7ddd717c8a None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:12.813817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:12.814601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:12.814710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:12.815032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] PATH_INFO: `/v3/users/557900ee01b94680927121dee25c5d14` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:12.829858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] RBAC: Authorizing `identity:get_user(user_id=557900ee01b94680927121dee25c5d14)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:12.831316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:12.833661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 572/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:12 2022] GET /identity/v3/users/557900ee01b94680927121dee25c5d14 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:12.840411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:12.938126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:12.958976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98506f89-2562-4a5e-ab49-ed510c396e2b None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:12.996672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:12.997276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:12.997551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:12.997821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:13.069211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:13.070660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:13.154578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:13.176135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 572/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:27:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:27:13.182703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:13.265112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:13.289735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f589bc-d1ce-487c-84aa-a7c4b89132d4 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:13.325441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35187191-2018-4828-b1f8-bf9607359513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:13.326069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35187191-2018-4828-b1f8-bf9607359513 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:13.326343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35187191-2018-4828-b1f8-bf9607359513 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:13.326606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35187191-2018-4828-b1f8-bf9607359513 None admin] PATH_INFO: `/v3/projects/6944b012bca24fbaaf8157b63f45c87e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:13.327832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35187191-2018-4828-b1f8-bf9607359513 None admin] RBAC: Authorizing `identity:get_project(project_id=6944b012bca24fbaaf8157b63f45c87e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:13.329297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35187191-2018-4828-b1f8-bf9607359513 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:13.330926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 573/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:13 2022] GET /identity/v3/projects/6944b012bca24fbaaf8157b63f45c87e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:13.337577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:13.441532 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e7101-9f2a-40a5-be89-d7e1aae25862 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:13.480854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb175af-6aab-43f8-8516-2bbf6f3384e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:13.481456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb175af-6aab-43f8-8516-2bbf6f3384e8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:13.481725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb175af-6aab-43f8-8516-2bbf6f3384e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:13.481988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb175af-6aab-43f8-8516-2bbf6f3384e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:13.482706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb175af-6aab-43f8-8516-2bbf6f3384e8 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=6944b012bca24fbaaf8157b63f45c87e, user.id=557900ee01b94680927121dee25c5d14) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:13.483031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb175af-6aab-43f8-8516-2bbf6f3384e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:13.484482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb175af-6aab-43f8-8516-2bbf6f3384e8 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:13.490066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 573/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:27:13 2022] GET /identity/v3/role_assignments?user.id=557900ee01b94680927121dee25c5d14&scope.project.id=6944b012bca24fbaaf8157b63f45c87e&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:15.366158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7a67-cc8c-4ddb-a7e3-fef3cb7d8dbc None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:15.366964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7a67-cc8c-4ddb-a7e3-fef3cb7d8dbc None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:15.366964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7a67-cc8c-4ddb-a7e3-fef3cb7d8dbc None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:15.369129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 574/1147] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:15.374525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9844ec-b5e5-4b5a-b924-7708e860e1b1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:15.374946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9844ec-b5e5-4b5a-b924-7708e860e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:15.375035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9844ec-b5e5-4b5a-b924-7708e860e1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:15.394574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6f9844ec-b5e5-4b5a-b924-7708e860e1b1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:15.395819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6f9844ec-b5e5-4b5a-b924-7708e860e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:15.444210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:15.480818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 574/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:15.765759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7a67-cc8c-4ddb-a7e3-fef3cb7d8dbc None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:15.879025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7a67-cc8c-4ddb-a7e3-fef3cb7d8dbc None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:15.921136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:15.922125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:15.922563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:15.922977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:15.939840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:27:15.945592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:27:15.946612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:27:15.947478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 575/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:27:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:27:15.953083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9844ec-b5e5-4b5a-b924-7708e860e1b1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:16.054255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9844ec-b5e5-4b5a-b924-7708e860e1b1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:16.090846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:16.091436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:16.091735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:16.092008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:16.092656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:16.092952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:16.094319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:16.102839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 575/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:27:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:27:16.109694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:16.116306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:16.219551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc22f8-b102-4c44-9236-09f1eb1f1610 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:16.258004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:16.258620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:16.258892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:16.259150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] PATH_INFO: `/v3/users/557900ee01b94680927121dee25c5d14` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:16.260393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] RBAC: Authorizing `identity:get_user(user_id=557900ee01b94680927121dee25c5d14)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:16.261836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:16.263403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 576/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:16 2022] GET /identity/v3/users/557900ee01b94680927121dee25c5d14 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:16.272333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:16.319999 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:16.381221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380d0e2-7999-406d-a466-44c6c0a5dd75 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:16.416417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e92e99-8c7f-4a31-a7d6-ddbd1dfad50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:16.417045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e92e99-8c7f-4a31-a7d6-ddbd1dfad50d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:16.417321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e92e99-8c7f-4a31-a7d6-ddbd1dfad50d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:16.417588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e92e99-8c7f-4a31-a7d6-ddbd1dfad50d None admin] PATH_INFO: `/v3/projects/6944b012bca24fbaaf8157b63f45c87e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:16.418739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e92e99-8c7f-4a31-a7d6-ddbd1dfad50d None admin] RBAC: Authorizing `identity:get_project(project_id=6944b012bca24fbaaf8157b63f45c87e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:16.420221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e92e99-8c7f-4a31-a7d6-ddbd1dfad50d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:16.421854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 576/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:16 2022] GET /identity/v3/projects/6944b012bca24fbaaf8157b63f45c87e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:16.431642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:16.519689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:16.551789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f14ed-049b-4277-bce8-75dc4e7f0371 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:16.587432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b64b9-c15e-4dc2-acd8-aaa9e351c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:16.588040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b64b9-c15e-4dc2-acd8-aaa9e351c553 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:16.588310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b64b9-c15e-4dc2-acd8-aaa9e351c553 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:16.588578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b64b9-c15e-4dc2-acd8-aaa9e351c553 None admin] PATH_INFO: `/v3/projects/6944b012bca24fbaaf8157b63f45c87e/users/557900ee01b94680927121dee25c5d14/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:16.590700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b64b9-c15e-4dc2-acd8-aaa9e351c553 None admin] RBAC: Authorizing `identity:create_grant(project_id=6944b012bca24fbaaf8157b63f45c87e, user_id=557900ee01b94680927121dee25c5d14, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:16.592174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b64b9-c15e-4dc2-acd8-aaa9e351c553 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:16.608655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 577/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:27:16 2022] PUT /identity/v3/projects/6944b012bca24fbaaf8157b63f45c87e/users/557900ee01b94680927121dee25c5d14/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:27:18.480361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3bd75-15d7-4a5a-8756-1fe9962ea2df None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:18.481268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3bd75-15d7-4a5a-8756-1fe9962ea2df None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:18.481268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3bd75-15d7-4a5a-8756-1fe9962ea2df None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:18.483343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 577/1154] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:18.489831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3292d-6df1-4b96-a622-075898983140 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:18.490264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3292d-6df1-4b96-a622-075898983140 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:18.490374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3292d-6df1-4b96-a622-075898983140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:18.515541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-12a3292d-6df1-4b96-a622-075898983140 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:18.517346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-12a3292d-6df1-4b96-a622-075898983140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:18.588806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 578/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:18.861072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3bd75-15d7-4a5a-8756-1fe9962ea2df None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:18.983251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3bd75-15d7-4a5a-8756-1fe9962ea2df None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:18.990711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:19.019284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:19.019936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:19.020287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:19.020548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:19.030989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:27:19.035607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:27:19.036701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:27:19.037931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 578/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:27:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:27:19.043544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3292d-6df1-4b96-a622-075898983140 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:19.112149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:19.152672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3292d-6df1-4b96-a622-075898983140 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:19.191939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:19.192708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:19.192998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:19.193277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:19.193947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:19.194251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:19.195648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:19.204662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 579/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:27:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:19.211110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:19.324198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f9bf7-2bfd-4c09-8a9c-02cdd5ec5c85 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:19.363843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:19.364525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:19.364673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:19.364956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] PATH_INFO: `/v3/users/557900ee01b94680927121dee25c5d14` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:19.366117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] RBAC: Authorizing `identity:get_user(user_id=557900ee01b94680927121dee25c5d14)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:19.367607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:19.369252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 579/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:19 2022] GET /identity/v3/users/557900ee01b94680927121dee25c5d14 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:19.375741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:19.478199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd63909-fbb5-44e9-ba41-bb055f8a672c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:19.501437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:19.513355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:19.513960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:19.514237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:19.514506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:19.582782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:19.584598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:19.695837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 580/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:27:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:27:19.702295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:19.738024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:19.808977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012007e8-2206-44f3-b0c8-6075f5f533c3 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:19.844306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb00abc-5b30-4afe-a586-1f507d22fe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:19.844929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb00abc-5b30-4afe-a586-1f507d22fe51 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:19.845201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb00abc-5b30-4afe-a586-1f507d22fe51 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:19.845430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb00abc-5b30-4afe-a586-1f507d22fe51 None admin] PATH_INFO: `/v3/projects/6944b012bca24fbaaf8157b63f45c87e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:19.847057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb00abc-5b30-4afe-a586-1f507d22fe51 None admin] RBAC: Authorizing `identity:get_project(project_id=6944b012bca24fbaaf8157b63f45c87e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:19.848533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb00abc-5b30-4afe-a586-1f507d22fe51 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:19.850185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 580/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:19 2022] GET /identity/v3/projects/6944b012bca24fbaaf8157b63f45c87e => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:19.857176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:19.885714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:19.960835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67c02-faaf-495f-aefa-d413a5b1a99b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:19.998236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5df97-c30f-4659-918a-3acad86bcf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:19.998827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5df97-c30f-4659-918a-3acad86bcf07 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:19.999101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5df97-c30f-4659-918a-3acad86bcf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:19.999368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5df97-c30f-4659-918a-3acad86bcf07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:20.000116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5df97-c30f-4659-918a-3acad86bcf07 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=6944b012bca24fbaaf8157b63f45c87e, user.id=557900ee01b94680927121dee25c5d14) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:20.000422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5df97-c30f-4659-918a-3acad86bcf07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:20.001854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5df97-c30f-4659-918a-3acad86bcf07 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:20.008542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 581/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:27:19 2022] GET /identity/v3/role_assignments?user.id=557900ee01b94680927121dee25c5d14&scope.project.id=6944b012bca24fbaaf8157b63f45c87e&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:21.911455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3c44-ae4c-41da-9a92-7adeabe63e81 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:21.911955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3c44-ae4c-41da-9a92-7adeabe63e81 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:21.912073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3c44-ae4c-41da-9a92-7adeabe63e81 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:21.914162 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 581/1162] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:21.922883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd786eb-7a04-4bc8-8395-418bbd81afa5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:21.923287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd786eb-7a04-4bc8-8395-418bbd81afa5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:21.923463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd786eb-7a04-4bc8-8395-418bbd81afa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:21.949714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5dd786eb-7a04-4bc8-8395-418bbd81afa5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:21.950923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5dd786eb-7a04-4bc8-8395-418bbd81afa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:22.023791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 582/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:22.298451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce3c44-ae4c-41da-9a92-7adeabe63e81 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:22.423027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce3c44-ae4c-41da-9a92-7adeabe63e81 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:22.424581 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:22.462881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:22.463489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:22.463807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:22.464078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:22.541057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:22.542510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:22.642124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 582/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:27:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:27:22.648742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd786eb-7a04-4bc8-8395-418bbd81afa5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:22.668398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:22.765189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd786eb-7a04-4bc8-8395-418bbd81afa5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:22.799333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58415860-f26c-4a07-8156-d2ee112aa5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:22.799993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415860-f26c-4a07-8156-d2ee112aa5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:22.800273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415860-f26c-4a07-8156-d2ee112aa5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:22.800546 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415860-f26c-4a07-8156-d2ee112aa5c7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:22.807824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58415860-f26c-4a07-8156-d2ee112aa5c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:22.809300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58415860-f26c-4a07-8156-d2ee112aa5c7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-58415860-f26c-4a07-8156-d2ee112aa5c7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:27:22.821776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:27:22.822944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 05:27:22.824100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:27:22.827747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 583/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:27:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:27:22.828937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:22.831201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:22.949537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e373ff-6188-4d9c-bf96-b4e9c6901503 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:22.983400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72377b68-05eb-4ca1-9a7a-bf854c074d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:22.984033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72377b68-05eb-4ca1-9a7a-bf854c074d59 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:22.984307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72377b68-05eb-4ca1-9a7a-bf854c074d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:22.984571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72377b68-05eb-4ca1-9a7a-bf854c074d59 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:22.985202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72377b68-05eb-4ca1-9a7a-bf854c074d59 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:22.985503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72377b68-05eb-4ca1-9a7a-bf854c074d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:22.986888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72377b68-05eb-4ca1-9a7a-bf854c074d59 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:22.994106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 583/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 22 05:27:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:24.886883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda95f5d-1ccf-4f7f-ab3d-c0168a7b0cdc None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:24.887449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda95f5d-1ccf-4f7f-ab3d-c0168a7b0cdc None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:24.887449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda95f5d-1ccf-4f7f-ab3d-c0168a7b0cdc None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:24.888797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 584/1167] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:24.895136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e110e-76c2-4e63-9098-4a0418917b8a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:24.895949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e110e-76c2-4e63-9098-4a0418917b8a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:24.896232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e110e-76c2-4e63-9098-4a0418917b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:24.916569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-933e110e-76c2-4e63-9098-4a0418917b8a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:24.917749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-933e110e-76c2-4e63-9098-4a0418917b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:24.984665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:24.993216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 584/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:25.256680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda95f5d-1ccf-4f7f-ab3d-c0168a7b0cdc None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:25.344288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:25.371137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda95f5d-1ccf-4f7f-ab3d-c0168a7b0cdc None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:25.406741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497672e4-4f25-4074-b411-91f49bed4862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:25.407340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497672e4-4f25-4074-b411-91f49bed4862 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:25.407619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497672e4-4f25-4074-b411-91f49bed4862 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:25.407898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497672e4-4f25-4074-b411-91f49bed4862 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:25.408555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497672e4-4f25-4074-b411-91f49bed4862 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:25.409942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497672e4-4f25-4074-b411-91f49bed4862 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:25.433560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 585/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Mon Aug 22 05:27:25 2022] POST /identity/v3/domains => generated 267 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:27:27.347035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4594cb0-4c9f-437e-8ea0-7d513f45c53b None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:27.347640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4594cb0-4c9f-437e-8ea0-7d513f45c53b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:27.347640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4594cb0-4c9f-437e-8ea0-7d513f45c53b None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:27.348993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 585/1170] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:27.355384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540691d-270c-4770-bf78-eaa378e2adfb None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:27.356185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540691d-270c-4770-bf78-eaa378e2adfb None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:27.356334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540691d-270c-4770-bf78-eaa378e2adfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:27.378350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-1540691d-270c-4770-bf78-eaa378e2adfb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:27.379612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1540691d-270c-4770-bf78-eaa378e2adfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:27.454694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 586/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:27.733887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4594cb0-4c9f-437e-8ea0-7d513f45c53b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:27.839858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4594cb0-4c9f-437e-8ea0-7d513f45c53b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:27.875182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ea84c-692b-4eeb-a78d-7f3b5fb6b0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:27.875855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ea84c-692b-4eeb-a78d-7f3b5fb6b0eb None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:27.876124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ea84c-692b-4eeb-a78d-7f3b5fb6b0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:27.876395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ea84c-692b-4eeb-a78d-7f3b5fb6b0eb None admin] PATH_INFO: `/v3/domains/746e7534558b48a9904a15ea7802bff8` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:27.888465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ea84c-692b-4eeb-a78d-7f3b5fb6b0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=746e7534558b48a9904a15ea7802bff8)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:27.889920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ea84c-692b-4eeb-a78d-7f3b5fb6b0eb None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:27.891475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 586/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Mon Aug 22 05:27:27 2022] GET /identity/v3/domains/746e7534558b48a9904a15ea7802bff8 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:27.897564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540691d-270c-4770-bf78-eaa378e2adfb None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:27.970969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:28.004654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540691d-270c-4770-bf78-eaa378e2adfb None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:28.038611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b876b34-cfcc-45f0-8ed8-060224790d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:28.039223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b876b34-cfcc-45f0-8ed8-060224790d99 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:28.039502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b876b34-cfcc-45f0-8ed8-060224790d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:28.039804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b876b34-cfcc-45f0-8ed8-060224790d99 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:28.040451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b876b34-cfcc-45f0-8ed8-060224790d99 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:28.041909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b876b34-cfcc-45f0-8ed8-060224790d99 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:28.078793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 587/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:27:27 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:27:29.988795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc20f-b865-4fd7-9852-d7c48d2ad8d0 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:29.989665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc20f-b865-4fd7-9852-d7c48d2ad8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:29.989665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffc20f-b865-4fd7-9852-d7c48d2ad8d0 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:29.991881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 587/1174] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:30.000992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad376-b4ef-4140-8ffe-54904bd7a40d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:30.001727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad376-b4ef-4140-8ffe-54904bd7a40d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:30.001891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad376-b4ef-4140-8ffe-54904bd7a40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:30.023717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-770ad376-b4ef-4140-8ffe-54904bd7a40d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:30.024915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-770ad376-b4ef-4140-8ffe-54904bd7a40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:30.103260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 588/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:30.380304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffc20f-b865-4fd7-9852-d7c48d2ad8d0 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:30.449761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:30.500686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffc20f-b865-4fd7-9852-d7c48d2ad8d0 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:30.537740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcd3a1-e7a9-4a8f-900f-7701c191fe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:30.538356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcd3a1-e7a9-4a8f-900f-7701c191fe95 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:30.538625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcd3a1-e7a9-4a8f-900f-7701c191fe95 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:30.538891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcd3a1-e7a9-4a8f-900f-7701c191fe95 None admin] PATH_INFO: `/v3/domains/746e7534558b48a9904a15ea7802bff8` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:30.540296 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcd3a1-e7a9-4a8f-900f-7701c191fe95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=746e7534558b48a9904a15ea7802bff8)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:30.541780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcd3a1-e7a9-4a8f-900f-7701c191fe95 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:30.543387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 588/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Mon Aug 22 05:27:30 2022] GET /identity/v3/domains/746e7534558b48a9904a15ea7802bff8 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:30.549665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ad376-b4ef-4140-8ffe-54904bd7a40d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:30.600775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:30.661874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ad376-b4ef-4140-8ffe-54904bd7a40d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:30.696531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565db7e-d185-4efe-8da4-8e6be4c9be2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:30.697135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565db7e-d185-4efe-8da4-8e6be4c9be2d None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:30.697369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565db7e-d185-4efe-8da4-8e6be4c9be2d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:30.697667 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565db7e-d185-4efe-8da4-8e6be4c9be2d None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:30.698306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565db7e-d185-4efe-8da4-8e6be4c9be2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:30.699783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565db7e-d185-4efe-8da4-8e6be4c9be2d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:30.723217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 589/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:27:30 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:27:32.639943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd60b6b-b2b8-443f-8b9e-70aa17403f66 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:32.640535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd60b6b-b2b8-443f-8b9e-70aa17403f66 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:32.640535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd60b6b-b2b8-443f-8b9e-70aa17403f66 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:32.641817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 589/1178] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:32.648387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86a45-65a3-4522-8182-b0388952dc34 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:32.649092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86a45-65a3-4522-8182-b0388952dc34 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:32.649268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86a45-65a3-4522-8182-b0388952dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:32.678572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-50d86a45-65a3-4522-8182-b0388952dc34 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:32.679799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-50d86a45-65a3-4522-8182-b0388952dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:32.756774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 590/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:33.019153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd60b6b-b2b8-443f-8b9e-70aa17403f66 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:33.130543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd60b6b-b2b8-443f-8b9e-70aa17403f66 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:33.154126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:33.166204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:33.166827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:33.167098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:33.167361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:33.177626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:27:33.182255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:27:33.183396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:27:33.184467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 590/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:27:33.192031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d86a45-65a3-4522-8182-b0388952dc34 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:33.226302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:33.303868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d86a45-65a3-4522-8182-b0388952dc34 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:33.340985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:33.341600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:33.341876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:33.342144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:33.342812 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:33.343111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:33.344535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:33.353714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 591/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:33.362703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:33.478063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeddafc-efef-4b97-a5b8-3044fcf5979a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:33.511114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:33.511802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:33.512076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:33.512342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] PATH_INFO: `/v3/users/753c8c52b3a34a23b7a2c1c592462e5e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:33.528517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] RBAC: Authorizing `identity:get_user(user_id=753c8c52b3a34a23b7a2c1c592462e5e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:33.529972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:33.531609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 591/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:33 2022] GET /identity/v3/users/753c8c52b3a34a23b7a2c1c592462e5e => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:27:33.537945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:33.646456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:33.650491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b3708-54e6-45e9-ba71-4b201e2b41c1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:33.686099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:33.686701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:33.686983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:33.687249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:33.759298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:33.760720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:33.869780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:33.874491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 592/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:27:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:27:33.880535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:33.927509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:33.985678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b7ddb-edc1-46bd-9ed3-e058f055a01f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:34.022072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de5d9e-01a1-4328-8826-ace96c0e241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:34.022623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de5d9e-01a1-4328-8826-ace96c0e241e None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:34.022907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de5d9e-01a1-4328-8826-ace96c0e241e None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:34.023179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de5d9e-01a1-4328-8826-ace96c0e241e None admin] PATH_INFO: `/v3/projects/2b60f2130eee40cda2de1146874e46a5` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:34.024367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de5d9e-01a1-4328-8826-ace96c0e241e None admin] RBAC: Authorizing `identity:get_project(project_id=2b60f2130eee40cda2de1146874e46a5)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:34.025822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de5d9e-01a1-4328-8826-ace96c0e241e None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:34.027460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 592/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:33 2022] GET /identity/v3/projects/2b60f2130eee40cda2de1146874e46a5 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:34.034524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:34.150226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7cb9-857b-402c-aa23-b15d80ddee91 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:34.189829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe6aa3-d01b-4949-85e6-79c201b14ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:34.190438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe6aa3-d01b-4949-85e6-79c201b14ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:34.190718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe6aa3-d01b-4949-85e6-79c201b14ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:34.190986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe6aa3-d01b-4949-85e6-79c201b14ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:34.192113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe6aa3-d01b-4949-85e6-79c201b14ae7 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=2b60f2130eee40cda2de1146874e46a5, user.id=753c8c52b3a34a23b7a2c1c592462e5e) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:34.192113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe6aa3-d01b-4949-85e6-79c201b14ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:34.193518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe6aa3-d01b-4949-85e6-79c201b14ae7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:34.199017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 593/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:27:34 2022] GET /identity/v3/role_assignments?user.id=753c8c52b3a34a23b7a2c1c592462e5e&scope.project.id=2b60f2130eee40cda2de1146874e46a5&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:36.061253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c99664-38ca-4065-95e3-aea6b07b8af8 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:36.061858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c99664-38ca-4065-95e3-aea6b07b8af8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:36.061858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c99664-38ca-4065-95e3-aea6b07b8af8 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:36.063133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 593/1186] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:36.069589 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7e729-00a5-4769-bb96-025f6a5c2e31 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:36.070293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7e729-00a5-4769-bb96-025f6a5c2e31 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:36.070476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7e729-00a5-4769-bb96-025f6a5c2e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:36.099217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-eec7e729-00a5-4769-bb96-025f6a5c2e31 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:36.100454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-eec7e729-00a5-4769-bb96-025f6a5c2e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:36.163116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:36.176159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 594/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:36.468402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c99664-38ca-4065-95e3-aea6b07b8af8 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:36.613956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c99664-38ca-4065-95e3-aea6b07b8af8 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:36.626260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:36.663702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:36.664472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:36.664973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:36.665464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:36.678428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:27:36.683088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:27:36.684173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:27:36.685041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 594/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:27:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:27:36.691595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7e729-00a5-4769-bb96-025f6a5c2e31 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:36.805639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7e729-00a5-4769-bb96-025f6a5c2e31 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:36.845589 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:36.846549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:36.846832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:36.847097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:36.848025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:36.848176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:36.850010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:36.859611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 595/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:27:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:36.868790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:36.997548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2dfa6b-6347-42ce-835d-a2ad99475bdf None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:37.049739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:37.050404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:37.050679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:37.051099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] PATH_INFO: `/v3/users/753c8c52b3a34a23b7a2c1c592462e5e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:37.053155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] RBAC: Authorizing `identity:get_user(user_id=753c8c52b3a34a23b7a2c1c592462e5e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:37.055077 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:37.057448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 595/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:36 2022] GET /identity/v3/users/753c8c52b3a34a23b7a2c1c592462e5e => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:37.063644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:37.133265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:37.184184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f46991-d0e4-491b-86e3-3f54728c6290 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:37.221065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112f498-7909-491b-915a-f7df370b912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:37.222057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112f498-7909-491b-915a-f7df370b912f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:37.222335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112f498-7909-491b-915a-f7df370b912f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:37.222605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112f498-7909-491b-915a-f7df370b912f None admin] PATH_INFO: `/v3/projects/2b60f2130eee40cda2de1146874e46a5` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:37.223837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112f498-7909-491b-915a-f7df370b912f None admin] RBAC: Authorizing `identity:get_project(project_id=2b60f2130eee40cda2de1146874e46a5)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:37.225290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112f498-7909-491b-915a-f7df370b912f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:37.226921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 596/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:37 2022] GET /identity/v3/projects/2b60f2130eee40cda2de1146874e46a5 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:37.233312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:37.275854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:37.348133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d861db-c332-4451-b60f-12038ba4b863 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:37.384774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d289c5-eb5f-409b-8f8d-03323c90ec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:37.385428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d289c5-eb5f-409b-8f8d-03323c90ec6c None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:37.385708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d289c5-eb5f-409b-8f8d-03323c90ec6c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:37.385980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d289c5-eb5f-409b-8f8d-03323c90ec6c None admin] PATH_INFO: `/v3/projects/2b60f2130eee40cda2de1146874e46a5/users/753c8c52b3a34a23b7a2c1c592462e5e/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:37.388011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d289c5-eb5f-409b-8f8d-03323c90ec6c None admin] RBAC: Authorizing `identity:create_grant(project_id=2b60f2130eee40cda2de1146874e46a5, user_id=753c8c52b3a34a23b7a2c1c592462e5e, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:37.389462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d289c5-eb5f-409b-8f8d-03323c90ec6c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:37.407187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 596/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:27:37 2022] PUT /identity/v3/projects/2b60f2130eee40cda2de1146874e46a5/users/753c8c52b3a34a23b7a2c1c592462e5e/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:27:39.284072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee14c3-334b-4a01-961f-1c2cea6a6f3d None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:39.284907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee14c3-334b-4a01-961f-1c2cea6a6f3d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:39.284907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee14c3-334b-4a01-961f-1c2cea6a6f3d None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:39.286987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 597/1193] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:27:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:27:39.293439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8e63-2646-4bec-9934-e451b0c4edf3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:39.294171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8e63-2646-4bec-9934-e451b0c4edf3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:39.294296 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8e63-2646-4bec-9934-e451b0c4edf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:39.316410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ddea8e63-2646-4bec-9934-e451b0c4edf3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:27:39.317595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ddea8e63-2646-4bec-9934-e451b0c4edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:27:39.397162 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 597/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:27:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:27:39.668213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee14c3-334b-4a01-961f-1c2cea6a6f3d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:39.778918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee14c3-334b-4a01-961f-1c2cea6a6f3d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:39.801901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:39.814800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:39.815418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:39.815724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:39.815987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:39.826908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:27:39.831497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:27:39.832707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:27:39.833593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 598/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:27:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:27:39.839049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8e63-2646-4bec-9934-e451b0c4edf3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:39.868922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:39.945765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8e63-2646-4bec-9934-e451b0c4edf3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:39.981414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:39.982010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:39.982285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:39.982547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:39.983204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:39.983500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:39.984900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:39.992940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 598/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:27:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:39.999008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:40.103609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc93da-97ae-4ed6-8bc4-1cd467cc2855 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:40.138222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:40.138835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:40.139106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:40.139375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] PATH_INFO: `/v3/users/753c8c52b3a34a23b7a2c1c592462e5e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:40.140645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] RBAC: Authorizing `identity:get_user(user_id=753c8c52b3a34a23b7a2c1c592462e5e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:40.142062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:40.143605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 599/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 22 05:27:39 2022] GET /identity/v3/users/753c8c52b3a34a23b7a2c1c592462e5e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:40.149810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:40.250257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:40.260694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1cf6-4e6c-4679-ae52-121bc0c82339 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:40.306701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:40.307389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:40.307618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:40.307883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:40.385125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:40.386579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:40.488218 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:40.499845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 599/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:27:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:27:40.506910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:40.564597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:40.626547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af74be-5528-4430-a698-892b7e3646ee None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:40.664557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08dcfe-dea4-4857-a734-8f930e26c861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:40.665168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08dcfe-dea4-4857-a734-8f930e26c861 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:40.665441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08dcfe-dea4-4857-a734-8f930e26c861 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:40.665707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08dcfe-dea4-4857-a734-8f930e26c861 None admin] PATH_INFO: `/v3/projects/2b60f2130eee40cda2de1146874e46a5` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:40.666937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08dcfe-dea4-4857-a734-8f930e26c861 None admin] RBAC: Authorizing `identity:get_project(project_id=2b60f2130eee40cda2de1146874e46a5)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:40.668419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08dcfe-dea4-4857-a734-8f930e26c861 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:40.670044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 600/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 22 05:27:40 2022] GET /identity/v3/projects/2b60f2130eee40cda2de1146874e46a5 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:40.676764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:27:40.777482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0c387-7539-48a6-9f6b-ba87fe757a2c None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:27:40.812622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad0a1d-0123-4559-a528-c381e80ed993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:27:40.813219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0a1d-0123-4559-a528-c381e80ed993 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:27:40.813498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0a1d-0123-4559-a528-c381e80ed993 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:27:40.813771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0a1d-0123-4559-a528-c381e80ed993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:27:40.814494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad0a1d-0123-4559-a528-c381e80ed993 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=2b60f2130eee40cda2de1146874e46a5, user.id=753c8c52b3a34a23b7a2c1c592462e5e) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:27:40.814802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad0a1d-0123-4559-a528-c381e80ed993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:27:40.816274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad0a1d-0123-4559-a528-c381e80ed993 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:27:40.822138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 600/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:27:40 2022] GET /identity/v3/role_assignments?user.id=753c8c52b3a34a23b7a2c1c592462e5e&scope.project.id=2b60f2130eee40cda2de1146874e46a5&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:27:50.659419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=41 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:27:50.819435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=73 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:36.864929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc4da4-7437-4a84-9274-145ff48d658b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:36.865622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc4da4-7437-4a84-9274-145ff48d658b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:36.865622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc4da4-7437-4a84-9274-145ff48d658b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:36.866897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 601/1201] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:28:36.873856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa36b6-7119-4527-a09f-b8bfe12cc3c3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:36.874515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa36b6-7119-4527-a09f-b8bfe12cc3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:36.874515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa36b6-7119-4527-a09f-b8bfe12cc3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:36.901244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-53fa36b6-7119-4527-a09f-b8bfe12cc3c3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:28:36.902637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-53fa36b6-7119-4527-a09f-b8bfe12cc3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:28:36.991622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 601/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:28:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:28:37.322066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc4da4-7437-4a84-9274-145ff48d658b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:37.485041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc4da4-7437-4a84-9274-145ff48d658b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:37.531246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:37.531927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:37.532208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:37.532474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:37.554393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:37.556686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:37.576502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:37.577720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:37.578831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:37.578831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:37.578831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:37.581330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 602/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:37.588425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa36b6-7119-4527-a09f-b8bfe12cc3c3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:37.735900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa36b6-7119-4527-a09f-b8bfe12cc3c3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:37.773692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:37.783451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99dfbc-1c76-4f5d-81ed-f1f81627912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:37.783451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99dfbc-1c76-4f5d-81ed-f1f81627912c None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:37.783451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99dfbc-1c76-4f5d-81ed-f1f81627912c None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:37.783451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99dfbc-1c76-4f5d-81ed-f1f81627912c None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:37.784312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99dfbc-1c76-4f5d-81ed-f1f81627912c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:37.784312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99dfbc-1c76-4f5d-81ed-f1f81627912c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:37.786035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99dfbc-1c76-4f5d-81ed-f1f81627912c None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:37.808929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 602/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:28:37.810233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:37.888049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:37.959025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fd436-3b2d-45d7-a24e-c5c673c7360d None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:38.000343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfb660-da29-44bd-98e6-d51a07465634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:38.001014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfb660-da29-44bd-98e6-d51a07465634 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:38.001305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfb660-da29-44bd-98e6-d51a07465634 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:38.001710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfb660-da29-44bd-98e6-d51a07465634 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:38.002271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfb660-da29-44bd-98e6-d51a07465634 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:38.003959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfb660-da29-44bd-98e6-d51a07465634 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:38.043607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 603/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 22 05:28:37 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:28:40.110342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bf4ca-0d94-4d7f-8d11-ac0f0a3e589a None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:40.110996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bf4ca-0d94-4d7f-8d11-ac0f0a3e589a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:40.110996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bf4ca-0d94-4d7f-8d11-ac0f0a3e589a None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:40.112267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 603/1206] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:28:40.118090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e313e-f8d4-491f-9ff7-dff5dc01b703 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:40.118515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e313e-f8d4-491f-9ff7-dff5dc01b703 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:40.118625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e313e-f8d4-491f-9ff7-dff5dc01b703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:40.137182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-be5e313e-f8d4-491f-9ff7-dff5dc01b703 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:28:40.138390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-be5e313e-f8d4-491f-9ff7-dff5dc01b703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:28:40.208625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 604/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:28:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:28:40.492478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bf4ca-0d94-4d7f-8d11-ac0f0a3e589a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:40.614232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bf4ca-0d94-4d7f-8d11-ac0f0a3e589a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:40.655305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:40.656061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:40.656361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:40.656697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:40.671839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:28:40.677423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:40.680393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:40.681231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 604/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:28:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:40.684996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e313e-f8d4-491f-9ff7-dff5dc01b703 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:40.731454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:40.813025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e313e-f8d4-491f-9ff7-dff5dc01b703 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:40.850921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:40.851749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:40.852146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:40.852534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:40.853348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:40.853959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:40.855513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:40.864725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 605/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:28:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:40.872237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:40.930013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:40.991043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e37c2-e42c-4b53-b24e-3d69ae28abd8 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:41.029220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:41.029892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:41.030199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:41.030507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:41.041514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:41.043272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:41.059122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:41.060387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:41.061304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:41.061304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:41.061304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:41.061304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 605/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:41.068459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:41.202550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:41.207307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560e17f-f602-44c1-8fc1-94b498e6fb0f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:41.247931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:41.248832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:41.249112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:41.249420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:41.250111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:41.250440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:41.252003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:41.269696 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 606/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:41.275973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:41.388609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ec6d9-c098-4f43-8dad-cdc0d73fa6b9 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:41.390227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:41.427934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:41.428772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:41.429145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:41.429506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:41.437549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:41.439009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:28:41.454495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:41.455872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:41.457766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:41.459740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 606/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:41.462183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:41.579781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c37272-8ad6-4999-9e42-070a67131fe1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:41.620176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:41.620844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:41.621179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:41.621446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:41.622230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:41.622579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:41.624329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:41.653256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 607/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 22 05:28:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:41.654538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:41.788457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097094a7-f804-4874-9324-9673fc0636bc None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:41.826091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:41.826829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:41.827210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:41.827560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:41.861873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:41.916146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:41.917703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:42.037579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 607/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:28:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 05:28:42.044117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:42.068419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:42.169504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7818e4-2631-42ce-8c6b-d1947fcaec7c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:42.210543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:42.211217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:42.211463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:42.211759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:42.223904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:42.225545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:42.241702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:42.242880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:42.243966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:42.243966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:42.243966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:42.243966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 608/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:42.249205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:42.322054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:42.362706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66ace5-9b65-4e21-88da-7fe113db6990 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:42.398477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:42.399115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:42.399390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:42.399690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:42.400335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:42.400641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:42.402169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:42.419767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 608/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:28:42.430286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:42.512679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:42.551180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d54fb-4006-4112-93b9-2cc5784191d8 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:42.591409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:42.592153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:42.592460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:42.592735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:42.601254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:42.602828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:28:42.614766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:42.615760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:42.616679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 609/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:28:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:42.621941 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:42.735836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81022c7d-8dd7-4a3f-a86c-add4897702cd None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:42.765157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:42.778295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a5d25-62b7-4981-b0cf-9af506125dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:42.778939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a5d25-62b7-4981-b0cf-9af506125dfe None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:42.779221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a5d25-62b7-4981-b0cf-9af506125dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:42.779495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a5d25-62b7-4981-b0cf-9af506125dfe None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:42.780259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a5d25-62b7-4981-b0cf-9af506125dfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:42.780546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a5d25-62b7-4981-b0cf-9af506125dfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:42.781981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a5d25-62b7-4981-b0cf-9af506125dfe None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:42.799974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 609/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:28:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:42.801669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:42.925900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e344b36-ba44-4caf-887b-88af74de0cfa None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:42.965833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:42.973224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b737d34-3cc0-44f4-8834-3689368d35db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:42.974104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b737d34-3cc0-44f4-8834-3689368d35db None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:42.974481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b737d34-3cc0-44f4-8834-3689368d35db None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:42.974808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b737d34-3cc0-44f4-8834-3689368d35db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:42.975541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b737d34-3cc0-44f4-8834-3689368d35db None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=733035fd9ee6414dbc23c69bb9206b6d) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:42.975895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b737d34-3cc0-44f4-8834-3689368d35db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:42.977499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b737d34-3cc0-44f4-8834-3689368d35db None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:42.983252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 610/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:28:42 2022] GET /identity/v3/role_assignments?user.id=733035fd9ee6414dbc23c69bb9206b6d&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:44.932617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79feaf-e5d5-445b-80bf-a7053b1e769b None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:44.933172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79feaf-e5d5-445b-80bf-a7053b1e769b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:44.933172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79feaf-e5d5-445b-80bf-a7053b1e769b None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:44.934540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 610/1220] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:28:44.940242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e6993-3a3b-4648-b943-0643115f1eb9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:44.940802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e6993-3a3b-4648-b943-0643115f1eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:44.940914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e6993-3a3b-4648-b943-0643115f1eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:44.959822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5d7e6993-3a3b-4648-b943-0643115f1eb9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:28:44.961207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5d7e6993-3a3b-4648-b943-0643115f1eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:28:45.047612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 611/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:28:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:28:45.319209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79feaf-e5d5-445b-80bf-a7053b1e769b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:45.441142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79feaf-e5d5-445b-80bf-a7053b1e769b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:45.480335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:45.481069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:45.481529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:45.481824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:45.492830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:28:45.497577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:45.498699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:45.499668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 611/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:45.505207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e6993-3a3b-4648-b943-0643115f1eb9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:45.614300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e6993-3a3b-4648-b943-0643115f1eb9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:45.638522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:45.654010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:45.654631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:45.654914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:45.655181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:45.655886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:45.656186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:45.657579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:45.666343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 612/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:45.672874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:45.723195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:45.787467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ed89a-2b62-4f87-a04f-b3cea5b0bc54 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:45.825128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:45.825833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:45.826155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:45.826498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:45.839363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:45.840899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:45.858078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:45.859302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:45.860347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:45.860347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:45.860347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:45.860347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 612/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:28:45.867327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:45.985287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0f62-0b15-48aa-a78a-0e43efd0eb94 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:46.021699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:46.022434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:46.022739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:46.023070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:46.023881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:46.024225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:46.025739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:46.039974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 613/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:46.046093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:46.145007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:46.153151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b591ca-156c-46e8-9871-6d3d29cde039 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:46.190571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:46.191345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:46.191683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:46.191957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:46.199640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:46.201117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:28:46.213030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:46.215800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:46.217025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 613/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:28:46.220910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:46.273526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:46.335743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178f88c-ccf3-4873-918b-8b8219aef4e2 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:46.377067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:46.377771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:46.378244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:46.379132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:46.380135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:46.380645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:46.382437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:46.403676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 614/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 22 05:28:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:46.411347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:46.536130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1e7ff-4178-4a32-ab57-565c3cdeb005 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:46.572955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:46.574467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:46.574467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:46.574467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:46.586192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:46.587735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:46.603419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:46.604458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:46.611008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:46.611008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:46.611008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:46.611008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 614/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:46.611352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:46.721694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16334-3a59-4606-a54a-b4c3414d6d47 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:46.724078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:46.757753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:46.758375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:46.758658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:46.759011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:46.759702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:46.760036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:46.761528 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:46.774101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 615/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:28:46.780072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:46.790417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:46.898560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d848db-9f43-4ca6-8ba6-74e4bc27f649 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:46.939423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:46.940094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:46.940372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:46.940685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:46.947974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:46.949468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:28:46.961270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:46.962582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:46.963777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 615/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:28:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:46.968572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:47.084430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c912a-5223-42b3-86c4-13ac2b04ccd1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:47.118833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fde6eb-060d-48c9-8719-21b978be0aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:47.119552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde6eb-060d-48c9-8719-21b978be0aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:47.119934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde6eb-060d-48c9-8719-21b978be0aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:47.120233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde6eb-060d-48c9-8719-21b978be0aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:47.121099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fde6eb-060d-48c9-8719-21b978be0aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:47.121466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fde6eb-060d-48c9-8719-21b978be0aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:47.123245 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fde6eb-060d-48c9-8719-21b978be0aa7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:47.138709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 616/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:28:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:47.145862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:47.226596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:47.264738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a48e34-9804-4984-a557-eb514a2b7969 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:47.301055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685376f-1eaa-48dc-adb1-df6aac3e50f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:47.301768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685376f-1eaa-48dc-adb1-df6aac3e50f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:47.302068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685376f-1eaa-48dc-adb1-df6aac3e50f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:47.302335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685376f-1eaa-48dc-adb1-df6aac3e50f3 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/733035fd9ee6414dbc23c69bb9206b6d/roles/be623ba761364dd593a441e1ca9cd988` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:47.320535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685376f-1eaa-48dc-adb1-df6aac3e50f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=733035fd9ee6414dbc23c69bb9206b6d, role_id=be623ba761364dd593a441e1ca9cd988)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:47.322302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685376f-1eaa-48dc-adb1-df6aac3e50f3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:47.336708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 616/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:28:47 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/733035fd9ee6414dbc23c69bb9206b6d/roles/be623ba761364dd593a441e1ca9cd988 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:28:49.401641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2905f-d676-43d5-847f-891c8fbbf959 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:49.402204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2905f-d676-43d5-847f-891c8fbbf959 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:49.402204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2905f-d676-43d5-847f-891c8fbbf959 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:49.403479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 617/1233] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:28:49.409000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682c81-163a-412d-97f2-be30fdc6bea1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:49.409656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682c81-163a-412d-97f2-be30fdc6bea1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:49.409834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682c81-163a-412d-97f2-be30fdc6bea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:49.428168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-20682c81-163a-412d-97f2-be30fdc6bea1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:28:49.429371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-20682c81-163a-412d-97f2-be30fdc6bea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:28:49.505064 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 617/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:28:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 05:28:49.771947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2905f-d676-43d5-847f-891c8fbbf959 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:49.798063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:49.877980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2905f-d676-43d5-847f-891c8fbbf959 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:49.916028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:49.916677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:49.916951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:49.917223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:49.928975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:28:49.934749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:49.935892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:49.936758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 618/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 22 05:28:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:28:49.942192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682c81-163a-412d-97f2-be30fdc6bea1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:49.992055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:50.052546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682c81-163a-412d-97f2-be30fdc6bea1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:50.091878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:50.092557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:50.092900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:50.093291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:50.094105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] RBAC: Adding query filter params (name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:50.094453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:50.096396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:50.106715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 618/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 22 05:28:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:50.112849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:50.210770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:50.230367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79822-db77-40dd-96e2-31bf96d424ec None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:50.268066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:50.268763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:50.269037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:50.269301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:50.280494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:50.281994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:50.298763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:50.299965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:50.301117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:50.301117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:50.301117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:50.301117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 619/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:50.306963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:50.417435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79204c31-b30c-460c-9dfa-c6c1f19cd71a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:50.428912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:50.459773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:50.460421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:50.460712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:50.460955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:50.461677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:50.462005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:50.463514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:50.476761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 619/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:50.483597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:50.590685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa4461-06e1-47fd-b586-38120f1f90b7 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:50.627757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:50.628441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:50.628697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:50.628961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:50.631856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:50.637283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:50.639129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:28:50.651128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:50.652306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:50.653427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 620/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:50.659043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:50.764157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895457-2264-46de-a3b7-eb0c5463292f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:50.798712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:50.799379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:50.799742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:50.800051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:50.800778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:50.801079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:50.802668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:50.828285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 620/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 22 05:28:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:50.836668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:50.947292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f4262-bab8-443b-b8eb-e0bfcc656595 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:50.985315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:50.985939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:50.986224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:50.986501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:51.054050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:51.062821 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:51.064344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:51.168035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 621/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:28:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:28:51.174075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:51.204095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:51.279697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e36a78-8ea4-4738-a7b7-10db0a1779e6 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:51.321897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:51.322865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:51.323433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:51.323747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:51.337567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:51.339029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:51.361334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:51.362953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:51.364379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:51.364379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:51.364379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:51.364379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 621/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:28:51.369450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:51.473519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:51.479480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc3056-b2d5-48e3-8f07-90dc05db07d1 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:51.516502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:51.517214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:51.517545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:51.517872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:51.518640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:51.518998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:51.520817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:51.534564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 622/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:51.542052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:51.628350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:51.652710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1404216-4495-49d3-8376-637a249a1060 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:51.695794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:51.696847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:51.697323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:51.697802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:51.705776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:51.707797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:28:51.722486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:51.723618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:51.724583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 622/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:28:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:28:51.729865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:51.833320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa14f0c-1cce-456a-bffd-2f54506ff20b None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:51.870655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a58ff-6fd7-4045-8203-bdc4d13a0b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:51.871271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a58ff-6fd7-4045-8203-bdc4d13a0b28 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:51.871544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a58ff-6fd7-4045-8203-bdc4d13a0b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:51.871840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a58ff-6fd7-4045-8203-bdc4d13a0b28 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:51.872558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a58ff-6fd7-4045-8203-bdc4d13a0b28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:51.872883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a58ff-6fd7-4045-8203-bdc4d13a0b28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:51.874303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a58ff-6fd7-4045-8203-bdc4d13a0b28 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:51.886859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 623/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:28:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:51.893077 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:52.003519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4a4e3-3f3b-4c27-8a78-96fda34fd61d None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:52.036681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c24a-905a-4d86-90b5-48e3dd4bdba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:52.037301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c24a-905a-4d86-90b5-48e3dd4bdba1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:52.037569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c24a-905a-4d86-90b5-48e3dd4bdba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:52.037833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c24a-905a-4d86-90b5-48e3dd4bdba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:52.038602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c24a-905a-4d86-90b5-48e3dd4bdba1 None admin] RBAC: Adding query filter params (role.id=be623ba761364dd593a441e1ca9cd988, scope.project.id=62d5a029139c4245a36acda923371996, user.id=733035fd9ee6414dbc23c69bb9206b6d) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:52.038890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c24a-905a-4d86-90b5-48e3dd4bdba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:52.040355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c24a-905a-4d86-90b5-48e3dd4bdba1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:52.046163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 623/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:28:51 2022] GET /identity/v3/role_assignments?user.id=733035fd9ee6414dbc23c69bb9206b6d&scope.project.id=62d5a029139c4245a36acda923371996&role.id=be623ba761364dd593a441e1ca9cd988 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:54.064306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d515275-f7fd-4661-b3d8-80cf5e855a4b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:54.064932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d515275-f7fd-4661-b3d8-80cf5e855a4b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:54.064932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d515275-f7fd-4661-b3d8-80cf5e855a4b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:54.066619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 624/1247] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:28:54.071840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c4af4-032b-4112-b4ea-199b1f35202b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:54.072292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c4af4-032b-4112-b4ea-199b1f35202b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:54.072411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c4af4-032b-4112-b4ea-199b1f35202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:54.078797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:54.092849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-b31c4af4-032b-4112-b4ea-199b1f35202b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:28:54.094288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b31c4af4-032b-4112-b4ea-199b1f35202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:28:54.170288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 624/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:28:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:28:54.440813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d515275-f7fd-4661-b3d8-80cf5e855a4b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:54.459818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:54.558223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d515275-f7fd-4661-b3d8-80cf5e855a4b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:54.599686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:54.600362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:54.600695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:54.600925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:54.611961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:28:54.616820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:54.617882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:54.618725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 625/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:54.624173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c4af4-032b-4112-b4ea-199b1f35202b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:54.744856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c4af4-032b-4112-b4ea-199b1f35202b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:54.781735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:54.788138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:54.788640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:54.788893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:54.789175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:54.789832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:54.790143 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:54.791500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:54.800619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 625/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:54.806479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:54.896102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:54.932258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fab267-7f2e-4358-9294-bbc71fefbe30 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:54.979990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:54.980630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:54.980920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:54.981202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:54.991555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:54.993112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:55.008564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:55.009747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:55.010856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:55.010856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:55.010856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:55.010856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 626/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:55.017403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:55.138971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88cf40-e84d-444a-ad8a-3930592d1dee None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:55.182102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:55.182733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:55.183015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:55.183294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:55.184108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:55.184455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:55.185936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:55.203766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 626/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:55.205273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:55.317540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a72a6c-3669-4e64-8efa-1bf696dfe179 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:55.349753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:55.358590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:55.359093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:55.359363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:55.359646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:55.368883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:55.370332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:28:55.382523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:55.383506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:55.385263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 627/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:55.390273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:55.452270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:55.503271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9af15-868b-45d9-82b2-7a94727bb738 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:55.541270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:55.541909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:55.542184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:55.542447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:55.543169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:55.543475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:55.544935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:55.562308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 627/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 22 05:28:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:55.569121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:55.682625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c157a6e-7a2b-40e5-9240-80065cb81777 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:55.719363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:55.720007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:55.720644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:55.720999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:55.795271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:55.810707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:55.812621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:55.930511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 628/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:28:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:28:55.936808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:56.048167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f9f96-98c2-4730-98fc-443a9cf61481 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:56.062459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:56.085232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:56.085872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:56.086151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:56.086414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:56.097780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:56.099313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:56.115966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:56.117271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:56.118601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:56.118601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:56.118601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:56.118601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 628/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:56.124399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:56.225612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:56.239661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d3d3b-9a66-4dc5-8c63-98ae159f2f17 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:56.277156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:56.277891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:56.278196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:56.278479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:56.279228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:56.279642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:56.281141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:56.293402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 629/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:56.301142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:56.424990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a856f3-8efd-4936-a583-5369f37ba6bb None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:56.461016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:56.461654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:56.461934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:56.462205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:56.468671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:56.470138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:28:56.483252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:56.484244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:28:56.485161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 629/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:56.491545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:56.612661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133675-9b80-4203-8d2f-efa961798d2e None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:56.650951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed1406-633e-47bc-8087-82ac1b2d5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:56.651565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed1406-633e-47bc-8087-82ac1b2d5668 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:56.651868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed1406-633e-47bc-8087-82ac1b2d5668 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:56.652140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed1406-633e-47bc-8087-82ac1b2d5668 None admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:56.652867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed1406-633e-47bc-8087-82ac1b2d5668 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:56.653163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed1406-633e-47bc-8087-82ac1b2d5668 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:56.654623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed1406-633e-47bc-8087-82ac1b2d5668 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:56.667755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 630/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:56.673952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:56.701242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:56.802611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c92488-dbde-4d8a-b870-cfbc66fdab83 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:56.840995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d0185-f1a0-4584-a145-0f22130c84c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:56.841613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d0185-f1a0-4584-a145-0f22130c84c3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:56.841909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d0185-f1a0-4584-a145-0f22130c84c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:56.842150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d0185-f1a0-4584-a145-0f22130c84c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:56.842893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d0185-f1a0-4584-a145-0f22130c84c3 None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=62d5a029139c4245a36acda923371996, user.id=733035fd9ee6414dbc23c69bb9206b6d) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:56.843195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d0185-f1a0-4584-a145-0f22130c84c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:56.844778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d0185-f1a0-4584-a145-0f22130c84c3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:56.851172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 630/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:28:56 2022] GET /identity/v3/role_assignments?user.id=733035fd9ee6414dbc23c69bb9206b6d&scope.project.id=62d5a029139c4245a36acda923371996&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:58.787794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58de1e3-ad57-4e5e-b727-30bb704e8fb7 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:58.788472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58de1e3-ad57-4e5e-b727-30bb704e8fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:58.788472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58de1e3-ad57-4e5e-b727-30bb704e8fb7 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:58.789668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 631/1261] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:28:58.794929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d247808-03a9-4d94-9299-4dfd888aa752 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:58.795363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d247808-03a9-4d94-9299-4dfd888aa752 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:58.795460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d247808-03a9-4d94-9299-4dfd888aa752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:58.815037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-1d247808-03a9-4d94-9299-4dfd888aa752 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:28:58.816336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1d247808-03a9-4d94-9299-4dfd888aa752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:28:58.885575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 631/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:28:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:28:59.151419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58de1e3-ad57-4e5e-b727-30bb704e8fb7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:59.166359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:59.264664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58de1e3-ad57-4e5e-b727-30bb704e8fb7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:59.299925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:59.300664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:59.301053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:59.301326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:59.313759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:28:59.319712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:59.320880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:59.321790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 632/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:28:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:59.327333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d247808-03a9-4d94-9299-4dfd888aa752 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:59.349133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:59.432594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d247808-03a9-4d94-9299-4dfd888aa752 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:59.468210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:59.468817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:59.469097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:59.469368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:59.470035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:59.470697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:59.472281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:59.480383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 632/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:28:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:59.489024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:59.560960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:59.598327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02c43f-faeb-4b69-8cdf-366c25868882 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:59.632198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:59.632822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:59.633099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:59.633381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:59.643955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:59.645434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:28:59.662419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:28:59.663529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:28:59.664606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:28:59.664606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:28:59.664606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:28:59.664606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 633/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:28:59.669689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:59.757496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:59.777496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ef9b6-1c75-4e40-8d57-02c67249fcea None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:59.812819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:59.813465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:59.813691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:59.813958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:59.814606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:28:59.814912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:59.816357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:28:59.828281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 633/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:28:59.834984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:28:59.945831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48086-376c-41b0-82bc-bff793798a19 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:28:59.965851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:28:59.983043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:28:59.983698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:28:59.983974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:28:59.984239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:28:59.992642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:28:59.994096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:29:00.005669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:00.006827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:29:00.008239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 634/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:28:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:00.013001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:00.126544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849664f-0fb8-46cf-8165-0ff1f2751892 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:00.163781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:00.164379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:00.164653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:00.164927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:00.165650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:00.165952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:00.167390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:00.176285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:00.185359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 634/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 22 05:29:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:00.192737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:00.305639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a1de-2178-4b81-8069-94d8ab07f69f None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:00.344094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:00.344815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:00.345159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:00.345824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:00.357166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:00.358638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:29:00.373239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:00.374376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:29:00.375400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:29:00.375400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:29:00.375400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:29:00.375400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 635/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 05:29:00.380364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:00.481832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5becdb3-239e-4bbe-9a37-639ae7312f2a None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:00.516627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:00.517238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:00.517514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:00.517788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:00.518460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:00.519138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:00.520385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:00.531845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 635/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:00.537849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:00.574254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:00.646045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78becfb-4334-4883-827a-3d48b6d10904 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:00.680246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:00.680858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:00.681122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:00.681379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:00.688040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:00.689521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:29:00.700463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:00.701488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:29:00.702519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:29:00.703749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 636/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:00.707945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:00.778646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:00.826626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292417e1-42da-4f32-9de9-2f80581ea914 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:00.863472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd94fb1-3e9d-47ce-b3f6-ef634c38240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:00.864113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd94fb1-3e9d-47ce-b3f6-ef634c38240b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:00.864394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd94fb1-3e9d-47ce-b3f6-ef634c38240b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:00.864661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd94fb1-3e9d-47ce-b3f6-ef634c38240b None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:00.865355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd94fb1-3e9d-47ce-b3f6-ef634c38240b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:00.865661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd94fb1-3e9d-47ce-b3f6-ef634c38240b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:00.867105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd94fb1-3e9d-47ce-b3f6-ef634c38240b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:00.879825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 636/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:00.886246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:00.980213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:00.990592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9030e-e602-45b9-86d7-916334dad723 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:01.026735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d81e9f-95e4-4e20-a5e2-cfade3ecd2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:01.027383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d81e9f-95e4-4e20-a5e2-cfade3ecd2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:01.027689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d81e9f-95e4-4e20-a5e2-cfade3ecd2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:01.027962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d81e9f-95e4-4e20-a5e2-cfade3ecd2f4 None admin] PATH_INFO: `/v3/projects/62d5a029139c4245a36acda923371996/users/733035fd9ee6414dbc23c69bb9206b6d/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:01.030092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d81e9f-95e4-4e20-a5e2-cfade3ecd2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=62d5a029139c4245a36acda923371996, user_id=733035fd9ee6414dbc23c69bb9206b6d, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:01.031554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d81e9f-95e4-4e20-a5e2-cfade3ecd2f4 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:01.050786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 637/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 22 05:29:00 2022] PUT /identity/v3/projects/62d5a029139c4245a36acda923371996/users/733035fd9ee6414dbc23c69bb9206b6d/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:29:02.911373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27aeab-52db-46cc-9bf4-618af5b5608d None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:02.911882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27aeab-52db-46cc-9bf4-618af5b5608d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:02.911990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27aeab-52db-46cc-9bf4-618af5b5608d None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:02.913350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 637/1274] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:29:02.918689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bacb5-f1e7-4f6c-a5ea-353876dbb16d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:02.919109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bacb5-f1e7-4f6c-a5ea-353876dbb16d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:02.919221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bacb5-f1e7-4f6c-a5ea-353876dbb16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:02.938825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4c1bacb5-f1e7-4f6c-a5ea-353876dbb16d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:29:02.940032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4c1bacb5-f1e7-4f6c-a5ea-353876dbb16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:29:03.014141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 638/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:29:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:29:03.292478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27aeab-52db-46cc-9bf4-618af5b5608d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:03.404657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27aeab-52db-46cc-9bf4-618af5b5608d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:03.422520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:03.440444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:03.441264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:03.441542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:03.441812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:03.452210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 05:29:03.456980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:03.458076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:03.458962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 638/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 22 05:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:03.464295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bacb5-f1e7-4f6c-a5ea-353876dbb16d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:03.546181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:03.569086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bacb5-f1e7-4f6c-a5ea-353876dbb16d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:03.603896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:03.604496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:03.604773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:03.605041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:03.605706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:03.606010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:03.607384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:03.615707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 639/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 22 05:29:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:03.621900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:03.724993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da482-634d-490c-9909-946f70bb08bb None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:03.760170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:03.760780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:03.761051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:03.761316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:03.773837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:03.775317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:29:03.792687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:03.793804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:29:03.795043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:29:03.795043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:29:03.795043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:03.795043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 639/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:03.800078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:03.918995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361b24f-3311-4bc9-9089-5c4b3c343e9c None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:03.956288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:03.956884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:03.957165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:03.957436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:03.958082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:03.958391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:03.959808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:03.973557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 640/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:03.982621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:04.018183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:04.103187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd7ea4-05c9-4c87-8e23-c18d0ad9db17 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:04.143195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:04.143903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:04.144184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:04.144440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:04.151752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:04.153218 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 05:29:04.164965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:04.166054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:04.167108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 640/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:29:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:04.174466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:04.179118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:04.290616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f3723-b232-48bc-b361-bb40be5c3e66 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:04.327627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:04.328372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:04.328505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:04.328777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:04.329609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:04.329818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:04.331591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:04.350006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 641/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 22 05:29:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:04.356801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:04.471897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:04.486221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382d064-0571-4cef-a211-a6b71cd84336 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:04.527723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:04.528328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:04.528610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:04.528881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:04.599132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:04.600623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:04.693211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:04.709115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 641/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 22 05:29:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 05:29:04.715341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:04.789466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:04.830206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2511-eb6a-4ed5-a636-da9448fd2fc5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:04.872585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:04.873213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:04.873487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:04.873755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:04.885153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:04.886633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 05:29:04.901842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:04.902936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 05:29:04.904078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 05:29:04.904078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 05:29:04.904078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:29:04.904078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 642/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:04.909566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:05.020539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02f5d0-1832-4ef1-942f-44703aea10bf None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:05.055015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:05.055619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:05.055902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:05.056173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:05.056805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:05.057102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:05.058468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:05.071125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 642/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 22 05:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:05.077012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:05.190936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d4fcd-698f-4c70-911f-3bb21b5d8199 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:05.206255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:05.229009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:05.229635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:05.229909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:05.230176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:05.236977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:05.238488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 05:29:05.249658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:05.250669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: ERROR keystone.server.flask.application Aug 22 05:29:05.251708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 643/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 22 05:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:05.257713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:05.323963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:05.377708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4cc3-22e1-4fa3-ae55-f28e8e685014 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:05.421397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1d837-9c9c-468b-af9c-d3442bda5de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:05.422261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1d837-9c9c-468b-af9c-d3442bda5de1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:05.422585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1d837-9c9c-468b-af9c-d3442bda5de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:05.422954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1d837-9c9c-468b-af9c-d3442bda5de1 None admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:05.424086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1d837-9c9c-468b-af9c-d3442bda5de1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:05.424478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1d837-9c9c-468b-af9c-d3442bda5de1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:05.426205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1d837-9c9c-468b-af9c-d3442bda5de1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:05.439897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 643/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 22 05:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:05.446472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:05.551966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679c25f-72f2-4c26-9ff2-0241ef4e14c5 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:05.589054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0a29b-621b-403d-8f9b-a239c73efefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:05.589557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0a29b-621b-403d-8f9b-a239c73efefb None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:05.589838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0a29b-621b-403d-8f9b-a239c73efefb None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:05.590105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0a29b-621b-403d-8f9b-a239c73efefb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:05.590826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0a29b-621b-403d-8f9b-a239c73efefb None admin] RBAC: Adding query filter params (role.id=6d5475aef2bb4ef98d9ff171fea3b73a, scope.project.id=62d5a029139c4245a36acda923371996, user.id=733035fd9ee6414dbc23c69bb9206b6d) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:05.591130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0a29b-621b-403d-8f9b-a239c73efefb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:05.592594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0a29b-621b-403d-8f9b-a239c73efefb None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:05.598792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 644/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 22 05:29:05 2022] GET /identity/v3/role_assignments?user.id=733035fd9ee6414dbc23c69bb9206b6d&scope.project.id=62d5a029139c4245a36acda923371996&role.id=6d5475aef2bb4ef98d9ff171fea3b73a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:07.490605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4619ae-2963-43b6-b6ef-0b10648d37b7 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:07.491165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4619ae-2963-43b6-b6ef-0b10648d37b7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:07.491165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4619ae-2963-43b6-b6ef-0b10648d37b7 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:07.492500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 644/1288] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:29:07.498832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc6f0-7d26-4268-a318-36dfcf683e1e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:07.499649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc6f0-7d26-4268-a318-36dfcf683e1e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:07.499810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc6f0-7d26-4268-a318-36dfcf683e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:07.519872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b44cc6f0-7d26-4268-a318-36dfcf683e1e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:29:07.521082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b44cc6f0-7d26-4268-a318-36dfcf683e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:29:07.551930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:07.594079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 645/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:29:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:29:07.856794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4619ae-2963-43b6-b6ef-0b10648d37b7 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:07.963692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4619ae-2963-43b6-b6ef-0b10648d37b7 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:07.974648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:08.002666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:08.003443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:08.003753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:08.004026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:08.004607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:08.005980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:08.016477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:08.017898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:08.019082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 645/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:29:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:08.025172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cc6f0-7d26-4268-a318-36dfcf683e1e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:08.127072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cc6f0-7d26-4268-a318-36dfcf683e1e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:08.162822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07387-2cec-45ce-b330-83816d934166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:08.163433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07387-2cec-45ce-b330-83816d934166 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:08.163740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07387-2cec-45ce-b330-83816d934166 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:08.164029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07387-2cec-45ce-b330-83816d934166 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:08.164648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07387-2cec-45ce-b330-83816d934166 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:08.164953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07387-2cec-45ce-b330-83816d934166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:08.166334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07387-2cec-45ce-b330-83816d934166 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:08.171225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 646/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 22 05:29:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:08.177903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:08.289106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f55b2-7a40-4c74-a60a-bbfd1b08b8c1 None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:08.332281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7d7f2-9bc0-42ab-a9a7-39623f8972c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:08.332935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d7f2-9bc0-42ab-a9a7-39623f8972c3 None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:08.333250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d7f2-9bc0-42ab-a9a7-39623f8972c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:08.333561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d7f2-9bc0-42ab-a9a7-39623f8972c3 None admin] PATH_INFO: `/v3/services` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:08.334238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d7f2-9bc0-42ab-a9a7-39623f8972c3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:08.334606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d7f2-9bc0-42ab-a9a7-39623f8972c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:08.336189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d7f2-9bc0-42ab-a9a7-39623f8972c3 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:08.341418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 646/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 22 05:29:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:10.198620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94fd4f-9f4c-42d2-b3b7-88a2f27e5ddf None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:10.199149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94fd4f-9f4c-42d2-b3b7-88a2f27e5ddf None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:10.199149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94fd4f-9f4c-42d2-b3b7-88a2f27e5ddf None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:10.200426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 647/1293] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:29:10.206968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acef6a9-56b9-476a-bdbf-6218bbf1c16e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:10.207759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acef6a9-56b9-476a-bdbf-6218bbf1c16e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:10.207899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acef6a9-56b9-476a-bdbf-6218bbf1c16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:10.238590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-2acef6a9-56b9-476a-bdbf-6218bbf1c16e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:29:10.239828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-2acef6a9-56b9-476a-bdbf-6218bbf1c16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:29:10.273596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:10.317192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 647/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:29:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:29:10.587942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94fd4f-9f4c-42d2-b3b7-88a2f27e5ddf None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:10.625110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:10.712559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94fd4f-9f4c-42d2-b3b7-88a2f27e5ddf None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:10.750336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4c24-aa3a-4dd1-93f0-18e88aaad779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:10.750935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4c24-aa3a-4dd1-93f0-18e88aaad779 None admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:10.751296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4c24-aa3a-4dd1-93f0-18e88aaad779 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:10.751614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4c24-aa3a-4dd1-93f0-18e88aaad779 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:10.752236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e4c24-aa3a-4dd1-93f0-18e88aaad779 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:10.753645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e4c24-aa3a-4dd1-93f0-18e88aaad779 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:10.772844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 648/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 22 05:29:10 2022] POST /identity/v3/services => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:29:12.669591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7671a-4dc0-45d4-8e47-413c93a7ac5f None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:12.670460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7671a-4dc0-45d4-8e47-413c93a7ac5f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:12.670608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7671a-4dc0-45d4-8e47-413c93a7ac5f None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:12.672791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 648/1296] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:29:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:29:12.678316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd241c1-cba1-4207-a596-a36c9d3c69b5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:12.678753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd241c1-cba1-4207-a596-a36c9d3c69b5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:12.678845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd241c1-cba1-4207-a596-a36c9d3c69b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:12.698090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5dd241c1-cba1-4207-a596-a36c9d3c69b5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:29:12.699269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5dd241c1-cba1-4207-a596-a36c9d3c69b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:29:12.774199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 649/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:29:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:29:13.034384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7671a-4dc0-45d4-8e47-413c93a7ac5f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:13.153949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7671a-4dc0-45d4-8e47-413c93a7ac5f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:13.197091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:13.197721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:13.197992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:13.198257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:13.198835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:13.200263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:13.209181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:13.210348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:13.211183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 649/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:29:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:13.216485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd241c1-cba1-4207-a596-a36c9d3c69b5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:13.238022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:13.331294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd241c1-cba1-4207-a596-a36c9d3c69b5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:13.370651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:13.371354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:13.371699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:13.372042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:13.372754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:13.373136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:13.374893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:13.381987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 650/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 22 05:29:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:13.388504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:13.425408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:13.496456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf88a3-3b5a-46fa-bd91-df4df47175ff None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:13.530801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8fe52-6879-4be9-b8e3-b0f368d4d79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:13.531375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8fe52-6879-4be9-b8e3-b0f368d4d79b None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:13.531698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8fe52-6879-4be9-b8e3-b0f368d4d79b None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:13.531968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8fe52-6879-4be9-b8e3-b0f368d4d79b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:13.532657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8fe52-6879-4be9-b8e3-b0f368d4d79b None admin] RBAC: Adding query filter params (interface=public, service_id=98163f24c5254841bbf41768a4e481ca, region_id=RegionOne) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:13.532964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8fe52-6879-4be9-b8e3-b0f368d4d79b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:13.534307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8fe52-6879-4be9-b8e3-b0f368d4d79b None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:13.539987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 650/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 22 05:29:13 2022] GET /identity/v3/endpoints?service_id=98163f24c5254841bbf41768a4e481ca&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:13.548479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:13.641040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:13.659322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f48018-d024-4845-bce8-8a237b7a1387 None admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:13.695594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb8e83-34b1-45c1-b18f-947120f45906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:13.696184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb8e83-34b1-45c1-b18f-947120f45906 None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:13.696473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb8e83-34b1-45c1-b18f-947120f45906 None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:13.696731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb8e83-34b1-45c1-b18f-947120f45906 None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:13.697330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb8e83-34b1-45c1-b18f-947120f45906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:13.698704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb8e83-34b1-45c1-b18f-947120f45906 None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:13.705354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 651/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 22 05:29:13 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:29:15.599483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca74c-3da1-4281-b3b4-618db14c5074 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:15.600459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca74c-3da1-4281-b3b4-618db14c5074 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:15.600459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca74c-3da1-4281-b3b4-618db14c5074 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:15.602465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 651/1302] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:29:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:29:15.608979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93de410-5f5b-4b6b-8055-ea2b6b1ea410 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:15.609663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93de410-5f5b-4b6b-8055-ea2b6b1ea410 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:15.609824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93de410-5f5b-4b6b-8055-ea2b6b1ea410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:15.635733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f93de410-5f5b-4b6b-8055-ea2b6b1ea410 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:29:15.636920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f93de410-5f5b-4b6b-8055-ea2b6b1ea410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:29:15.714723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 652/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:29:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 05:29:15.983529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca74c-3da1-4281-b3b4-618db14c5074 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:16.095542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:16.103902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca74c-3da1-4281-b3b4-618db14c5074 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:16.138912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:16.139534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:16.139832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:16.140097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:16.140637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:16.142037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 05:29:16.154627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 05:29:16.155763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application with Lock( Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self._enter() Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application return self.creator() Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application created_value = creator( Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: ERROR keystone.server.flask.application Aug 22 05:29:16.156898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 652/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 22 05:29:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 05:29:16.164503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93de410-5f5b-4b6b-8055-ea2b6b1ea410 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:16.252674 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:16.276759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93de410-5f5b-4b6b-8055-ea2b6b1ea410 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:16.312313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71842223-036b-43c1-9829-b0a785e13d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:16.312957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842223-036b-43c1-9829-b0a785e13d6d None admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:16.313219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842223-036b-43c1-9829-b0a785e13d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:16.313451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842223-036b-43c1-9829-b0a785e13d6d None admin] PATH_INFO: `/v3/services` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:16.314083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842223-036b-43c1-9829-b0a785e13d6d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:29:16.314378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842223-036b-43c1-9829-b0a785e13d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:16.315782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842223-036b-43c1-9829-b0a785e13d6d None admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:16.321658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 653/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 22 05:29:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:29:16.327861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:29:16.435254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31537c-a8d5-4275-a442-f909625a204f None admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:29:16.470511 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db4598-0ace-44f1-b67f-8f1ff2f327f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:29:16.471113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db4598-0ace-44f1-b67f-8f1ff2f327f1 None admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:29:16.471391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db4598-0ace-44f1-b67f-8f1ff2f327f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:29:16.471682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db4598-0ace-44f1-b67f-8f1ff2f327f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:29:16.472355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db4598-0ace-44f1-b67f-8f1ff2f327f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:29:16.473772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db4598-0ace-44f1-b67f-8f1ff2f327f1 None admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:29:16.496455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 653/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:29:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:29:26.318534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=30 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:29:26.492074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:21.287249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d1aff-e0f2-4b9f-ae03-7138dfa88b54 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:21.288335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d1aff-e0f2-4b9f-ae03-7138dfa88b54 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:21.288845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d1aff-e0f2-4b9f-ae03-7138dfa88b54 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:21.290408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 654/1307] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:21.296110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bd65d-9511-4879-91db-770913264803 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:21.296541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bd65d-9511-4879-91db-770913264803 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:21.296744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bd65d-9511-4879-91db-770913264803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:21.389589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-866bd65d-9511-4879-91db-770913264803 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:21.390810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-866bd65d-9511-4879-91db-770913264803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:21.519916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 654/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:21.829187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52f60-4d5b-45f2-a825-39570c51b4f6 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:21.829478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52f60-4d5b-45f2-a825-39570c51b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:21.829776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a52f60-4d5b-45f2-a825-39570c51b4f6 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:21.832265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 655/1309] 10.222.0.18 () {54 vars in 965 bytes} [Mon Aug 22 05:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:21.842116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a54e31-1232-4487-81c2-502eb268c667 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:21.842824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a54e31-1232-4487-81c2-502eb268c667 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:21.843336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a54e31-1232-4487-81c2-502eb268c667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:21.949460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-28a54e31-1232-4487-81c2-502eb268c667 None None] MFA Rules not processed for user `0a5c594aafc14c1e86ffa9702f9be964`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:21.950759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-28a54e31-1232-4487-81c2-502eb268c667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:21.972919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:22.035097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 655/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Aug 22 05:30:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:22.044218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a52f60-4d5b-45f2-a825-39570c51b4f6 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:22.045857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-96a52f60-4d5b-45f2-a825-39570c51b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:22.157886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a52f60-4d5b-45f2-a825-39570c51b4f6 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:22.255012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d5cf5-c937-451d-87f9-d68836c94a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:22.256282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d5cf5-c937-451d-87f9-d68836c94a30 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:22.256718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d5cf5-c937-451d-87f9-d68836c94a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:22.257243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d5cf5-c937-451d-87f9-d68836c94a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:22.259163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-482d5cf5-c937-451d-87f9-d68836c94a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:22.328226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:22.392252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d5cf5-c937-451d-87f9-d68836c94a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:22.393725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d5cf5-c937-451d-87f9-d68836c94a30 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:22.416688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 656/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:24.620973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad55a74-8594-4a2e-aaa8-15f731e511e8 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:24.621540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad55a74-8594-4a2e-aaa8-15f731e511e8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:24.621540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad55a74-8594-4a2e-aaa8-15f731e511e8 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:24.622838 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 656/1312] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:24.628306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61941-1007-4ef1-8ad7-8be25c15970d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:24.628714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61941-1007-4ef1-8ad7-8be25c15970d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:24.628823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61941-1007-4ef1-8ad7-8be25c15970d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:24.713842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f5e61941-1007-4ef1-8ad7-8be25c15970d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:24.715083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f5e61941-1007-4ef1-8ad7-8be25c15970d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:24.823880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 657/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:25.117779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf992-1eb2-4d31-b503-79e794e42b38 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:25.118132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf992-1eb2-4d31-b503-79e794e42b38 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:25.118678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf992-1eb2-4d31-b503-79e794e42b38 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:25.120442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 657/1314] 10.222.0.18 () {54 vars in 965 bytes} [Mon Aug 22 05:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:25.126473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78659-d530-4faf-a60b-08573e8890cf None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:25.126765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78659-d530-4faf-a60b-08573e8890cf None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:25.127046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78659-d530-4faf-a60b-08573e8890cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:25.139124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:25.204039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-89b78659-d530-4faf-a60b-08573e8890cf None None] MFA Rules not processed for user `0a5c594aafc14c1e86ffa9702f9be964`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:25.205171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-89b78659-d530-4faf-a60b-08573e8890cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:25.286926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 658/1315] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Aug 22 05:30:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:25.296052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf992-1eb2-4d31-b503-79e794e42b38 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:25.297592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4bdbf992-1eb2-4d31-b503-79e794e42b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:25.409405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf992-1eb2-4d31-b503-79e794e42b38 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:25.486475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:25.489476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a1484-eb3b-45aa-b003-0e4bf5510d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:25.490170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a1484-eb3b-45aa-b003-0e4bf5510d78 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:25.490459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a1484-eb3b-45aa-b003-0e4bf5510d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:25.490745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a1484-eb3b-45aa-b003-0e4bf5510d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:25.492976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a96a1484-eb3b-45aa-b003-0e4bf5510d78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:25.612289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a1484-eb3b-45aa-b003-0e4bf5510d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:25.613781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a1484-eb3b-45aa-b003-0e4bf5510d78 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:25.632492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 658/1316] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:28.062024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3f0b-c024-4d4b-8182-026aca5773bc None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:28.062572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3f0b-c024-4d4b-8182-026aca5773bc None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:28.062572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3f0b-c024-4d4b-8182-026aca5773bc None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:28.063945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 659/1317] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:28.069616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb390-1e04-46a2-886e-5a0e708eabae None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:28.070122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb390-1e04-46a2-886e-5a0e708eabae None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:28.070122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb390-1e04-46a2-886e-5a0e708eabae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:28.137636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9c1bb390-1e04-46a2-886e-5a0e708eabae None None] MFA Rules not processed for user `f27b32aa767849b1b542bbb8f3eeeb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:28.138828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9c1bb390-1e04-46a2-886e-5a0e708eabae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:28.191363 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:28.241292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 659/1318] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:28.526653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b3f0b-c024-4d4b-8182-026aca5773bc None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:28.528240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3a5b3f0b-c024-4d4b-8182-026aca5773bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:28.637085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b3f0b-c024-4d4b-8182-026aca5773bc None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:28.640002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:28.727065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86997d-e60e-4d33-8369-ee4d97324ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:28.727740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86997d-e60e-4d33-8369-ee4d97324ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:28.728002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86997d-e60e-4d33-8369-ee4d97324ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:28.728298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86997d-e60e-4d33-8369-ee4d97324ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:28.729901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4d86997d-e60e-4d33-8369-ee4d97324ff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:28.839147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86997d-e60e-4d33-8369-ee4d97324ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:28.840690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86997d-e60e-4d33-8369-ee4d97324ff0 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:28.859066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 660/1319] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:31.369918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8e89-622e-4eef-a9b6-0779baa682d1 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:31.370488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8e89-622e-4eef-a9b6-0779baa682d1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:31.370488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8e89-622e-4eef-a9b6-0779baa682d1 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:31.371782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 660/1320] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:31.377304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78347c7-d98a-48e8-ac30-d566bcc009ed None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:31.377753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78347c7-d98a-48e8-ac30-d566bcc009ed None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:31.377858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78347c7-d98a-48e8-ac30-d566bcc009ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:31.421143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:31.453296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f78347c7-d98a-48e8-ac30-d566bcc009ed None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:31.453296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f78347c7-d98a-48e8-ac30-d566bcc009ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:31.560622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 661/1321] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:31.861027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f8e89-622e-4eef-a9b6-0779baa682d1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:31.862592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ac4f8e89-622e-4eef-a9b6-0779baa682d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:31.972243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f8e89-622e-4eef-a9b6-0779baa682d1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:32.047375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c99f9-a76d-4942-a132-40034e7a65b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:32.048005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c99f9-a76d-4942-a132-40034e7a65b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:32.048279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c99f9-a76d-4942-a132-40034e7a65b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:32.048543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c99f9-a76d-4942-a132-40034e7a65b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:32.050142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-376c99f9-a76d-4942-a132-40034e7a65b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:32.058489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:32.163892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c99f9-a76d-4942-a132-40034e7a65b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:32.165335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c99f9-a76d-4942-a132-40034e7a65b6 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:32.184407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 661/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:35.038850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce1ee-238c-4cfa-8480-762037f08639 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:35.039387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce1ee-238c-4cfa-8480-762037f08639 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:35.039387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce1ee-238c-4cfa-8480-762037f08639 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:35.040918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 662/1323] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:35.046035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b04d-f73f-4874-8b3e-903979d1333e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:35.046519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b04d-f73f-4874-8b3e-903979d1333e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:35.046519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b04d-f73f-4874-8b3e-903979d1333e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:35.121916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a616b04d-f73f-4874-8b3e-903979d1333e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:35.123158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a616b04d-f73f-4874-8b3e-903979d1333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:35.183521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:35.227624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 662/1324] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:35.515595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce1ee-238c-4cfa-8480-762037f08639 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:35.517182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-142ce1ee-238c-4cfa-8480-762037f08639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:35.623438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce1ee-238c-4cfa-8480-762037f08639 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:35.636078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:35.696629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b0613-234f-4adb-b1d3-9be5ffe813ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:35.697341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0613-234f-4adb-b1d3-9be5ffe813ef service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:35.697690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0613-234f-4adb-b1d3-9be5ffe813ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:35.697976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0613-234f-4adb-b1d3-9be5ffe813ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:35.699602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-426b0613-234f-4adb-b1d3-9be5ffe813ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:35.815171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b0613-234f-4adb-b1d3-9be5ffe813ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:35.816768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b0613-234f-4adb-b1d3-9be5ffe813ef service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:35.836055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 663/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:37.908220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de11c5b-ee0b-49bc-a17d-9d048e318141 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:37.908709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de11c5b-ee0b-49bc-a17d-9d048e318141 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:37.908809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de11c5b-ee0b-49bc-a17d-9d048e318141 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:37.910777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 663/1326] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:37.916810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823214f-9caa-4ada-88ce-5c0c9992d759 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:37.917342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823214f-9caa-4ada-88ce-5c0c9992d759 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:37.917342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823214f-9caa-4ada-88ce-5c0c9992d759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:37.979040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:37.998893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9823214f-9caa-4ada-88ce-5c0c9992d759 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:38.000196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9823214f-9caa-4ada-88ce-5c0c9992d759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:38.113237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 664/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:38.404012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de11c5b-ee0b-49bc-a17d-9d048e318141 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:38.405557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5de11c5b-ee0b-49bc-a17d-9d048e318141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:38.513835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de11c5b-ee0b-49bc-a17d-9d048e318141 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:38.598627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac46e4-4e99-4679-b4f3-a3d3af11ede5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:38.599249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac46e4-4e99-4679-b4f3-a3d3af11ede5 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:38.599528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac46e4-4e99-4679-b4f3-a3d3af11ede5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:38.599830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac46e4-4e99-4679-b4f3-a3d3af11ede5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:38.601435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-62ac46e4-4e99-4679-b4f3-a3d3af11ede5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:38.618681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:38.714217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac46e4-4e99-4679-b4f3-a3d3af11ede5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:38.715697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac46e4-4e99-4679-b4f3-a3d3af11ede5 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:38.736112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 664/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:40.698200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f9f86-02f7-4b66-839c-1732d6917dd3 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:40.698740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f9f86-02f7-4b66-839c-1732d6917dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:40.698740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f9f86-02f7-4b66-839c-1732d6917dd3 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:40.700100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 665/1329] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:40.705560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f6976-bb4b-4d49-8a80-aacae9a506d2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:40.706011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f6976-bb4b-4d49-8a80-aacae9a506d2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:40.706113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f6976-bb4b-4d49-8a80-aacae9a506d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:40.781654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-8d8f6976-bb4b-4d49-8a80-aacae9a506d2 None None] MFA Rules not processed for user `f27b32aa767849b1b542bbb8f3eeeb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:40.782875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8d8f6976-bb4b-4d49-8a80-aacae9a506d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:40.851793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:40.884319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 665/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:41.166759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f9f86-02f7-4b66-839c-1732d6917dd3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:41.168432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-269f9f86-02f7-4b66-839c-1732d6917dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:41.284755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f9f86-02f7-4b66-839c-1732d6917dd3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:41.305719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:41.364464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c5efe-a4c4-4739-9f12-aeeb0cdf2fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:41.365078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c5efe-a4c4-4739-9f12-aeeb0cdf2fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:41.365358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c5efe-a4c4-4739-9f12-aeeb0cdf2fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:41.365627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c5efe-a4c4-4739-9f12-aeeb0cdf2fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:41.367232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-923c5efe-a4c4-4739-9f12-aeeb0cdf2fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:41.481527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c5efe-a4c4-4739-9f12-aeeb0cdf2fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:41.482900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c5efe-a4c4-4739-9f12-aeeb0cdf2fe5 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:41.501836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 666/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:43.738108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da971646-6a0d-49bf-8ce4-6a4404a00dfa None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:43.738908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da971646-6a0d-49bf-8ce4-6a4404a00dfa None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:43.739193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da971646-6a0d-49bf-8ce4-6a4404a00dfa None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:43.740673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 666/1332] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:43.747218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a8b42-c319-4f32-ab07-bfd33cccf1bc None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:43.748030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a8b42-c319-4f32-ab07-bfd33cccf1bc None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:43.748208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a8b42-c319-4f32-ab07-bfd33cccf1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:43.820503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:43.824366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-558a8b42-c319-4f32-ab07-bfd33cccf1bc None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:43.827182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-558a8b42-c319-4f32-ab07-bfd33cccf1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:43.930928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 667/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:44.228093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da971646-6a0d-49bf-8ce4-6a4404a00dfa None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:44.230501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-da971646-6a0d-49bf-8ce4-6a4404a00dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:44.341632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da971646-6a0d-49bf-8ce4-6a4404a00dfa None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:44.422538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5583b-c8c7-445f-91db-7e43576235de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:44.423146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5583b-c8c7-445f-91db-7e43576235de service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:44.423417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5583b-c8c7-445f-91db-7e43576235de service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:44.423720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5583b-c8c7-445f-91db-7e43576235de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:44.425311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aca5583b-c8c7-445f-91db-7e43576235de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:44.458066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:44.546502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5583b-c8c7-445f-91db-7e43576235de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:44.547869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5583b-c8c7-445f-91db-7e43576235de service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:44.567793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 667/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:46.528528 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaf688-3102-4238-a712-3da662a19f54 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:46.529220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaf688-3102-4238-a712-3da662a19f54 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:46.529220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaf688-3102-4238-a712-3da662a19f54 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:46.530412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 668/1335] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:46.537056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7442b01-6d9b-4792-bbad-92c396d0c748 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:46.537840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7442b01-6d9b-4792-bbad-92c396d0c748 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:46.537991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7442b01-6d9b-4792-bbad-92c396d0c748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:46.623977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f7442b01-6d9b-4792-bbad-92c396d0c748 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:46.625180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f7442b01-6d9b-4792-bbad-92c396d0c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:46.708037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:46.732541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 668/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:47.015536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaf688-3102-4238-a712-3da662a19f54 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:47.017142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-95eaf688-3102-4238-a712-3da662a19f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:47.124525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaf688-3102-4238-a712-3da662a19f54 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:47.154023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:47.203753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae266-27ad-423b-be74-2158c01e3583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:47.204369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae266-27ad-423b-be74-2158c01e3583 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:47.204638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae266-27ad-423b-be74-2158c01e3583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:47.204899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae266-27ad-423b-be74-2158c01e3583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:47.206498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1f6ae266-27ad-423b-be74-2158c01e3583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:47.321495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae266-27ad-423b-be74-2158c01e3583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:47.322935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ae266-27ad-423b-be74-2158c01e3583 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:47.349599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 669/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:49.677965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c164f8-495d-4b75-a1eb-840abe1ad1d8 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:49.678576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c164f8-495d-4b75-a1eb-840abe1ad1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:49.678576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c164f8-495d-4b75-a1eb-840abe1ad1d8 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:49.679910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 669/1338] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:49.686295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012428b-25fe-477c-a8b3-ed30469084c6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:49.687061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012428b-25fe-477c-a8b3-ed30469084c6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:49.687222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012428b-25fe-477c-a8b3-ed30469084c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:49.765178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f012428b-25fe-477c-a8b3-ed30469084c6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:49.766555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f012428b-25fe-477c-a8b3-ed30469084c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:49.775622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:49.873600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 670/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:50.169013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c164f8-495d-4b75-a1eb-840abe1ad1d8 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:50.171528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-31c164f8-495d-4b75-a1eb-840abe1ad1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:50.295234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c164f8-495d-4b75-a1eb-840abe1ad1d8 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:50.384669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3408aed-a5f6-4500-b228-9a0485e55c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:50.385277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408aed-a5f6-4500-b228-9a0485e55c33 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:50.385551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408aed-a5f6-4500-b228-9a0485e55c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:50.385822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408aed-a5f6-4500-b228-9a0485e55c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:50.387409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c3408aed-a5f6-4500-b228-9a0485e55c33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:50.430430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:50.520481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3408aed-a5f6-4500-b228-9a0485e55c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:50.521902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3408aed-a5f6-4500-b228-9a0485e55c33 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:50.541163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 670/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:54.397981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40652853-d265-494b-ba55-2a783e9ebf99 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:54.398426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40652853-d265-494b-ba55-2a783e9ebf99 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:54.398517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40652853-d265-494b-ba55-2a783e9ebf99 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:54.400060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 671/1341] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:54.405454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38e454-51f6-4124-a979-daadaf741e90 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:54.405903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38e454-51f6-4124-a979-daadaf741e90 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:54.405997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38e454-51f6-4124-a979-daadaf741e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:54.474106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ae38e454-51f6-4124-a979-daadaf741e90 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:54.475357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ae38e454-51f6-4124-a979-daadaf741e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:54.569972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:54.582764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 671/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:54.864971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40652853-d265-494b-ba55-2a783e9ebf99 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:54.866605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-40652853-d265-494b-ba55-2a783e9ebf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:54.983320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40652853-d265-494b-ba55-2a783e9ebf99 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:55.024075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:55.061334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9456c3-793d-4cfe-86b5-eecf08b60b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:55.061947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9456c3-793d-4cfe-86b5-eecf08b60b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:55.062197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9456c3-793d-4cfe-86b5-eecf08b60b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:55.062492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9456c3-793d-4cfe-86b5-eecf08b60b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:55.064130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2c9456c3-793d-4cfe-86b5-eecf08b60b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:55.178092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9456c3-793d-4cfe-86b5-eecf08b60b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:55.179566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9456c3-793d-4cfe-86b5-eecf08b60b99 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:55.198614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 672/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:30:57.700585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202077-da58-4b1f-b16a-a4e5f7a7f9da None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:57.701146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202077-da58-4b1f-b16a-a4e5f7a7f9da None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:57.701146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202077-da58-4b1f-b16a-a4e5f7a7f9da None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:57.702504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 672/1344] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:30:57.708468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd79e-2ef8-41b3-968b-de1ac5dd06c6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:57.709047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd79e-2ef8-41b3-968b-de1ac5dd06c6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:57.709047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd79e-2ef8-41b3-968b-de1ac5dd06c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:57.779434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-993fd79e-2ef8-41b3-968b-de1ac5dd06c6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:30:57.780676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-993fd79e-2ef8-41b3-968b-de1ac5dd06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:57.800917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:57.885827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 673/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:30:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:30:58.162621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f202077-da58-4b1f-b16a-a4e5f7a7f9da None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:30:58.164212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1f202077-da58-4b1f-b16a-a4e5f7a7f9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:58.274196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f202077-da58-4b1f-b16a-a4e5f7a7f9da None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:30:58.368973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d9965-7be9-4d5a-8223-241a69f67435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:30:58.369650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d9965-7be9-4d5a-8223-241a69f67435 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:30:58.369931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d9965-7be9-4d5a-8223-241a69f67435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:30:58.370203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d9965-7be9-4d5a-8223-241a69f67435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:30:58.371971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d67d9965-7be9-4d5a-8223-241a69f67435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:30:58.418756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:30:58.491417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d9965-7be9-4d5a-8223-241a69f67435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:30:58.492898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d9965-7be9-4d5a-8223-241a69f67435 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:30:58.514201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 673/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:30:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:31:02.588422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c996a-6ff5-4d1b-addd-34c533db19ef None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:02.589005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c996a-6ff5-4d1b-addd-34c533db19ef None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:02.589005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c996a-6ff5-4d1b-addd-34c533db19ef None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:02.590288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 674/1347] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:02.596252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf162d0-7869-4034-b3c8-892906a5cd5a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:02.596734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf162d0-7869-4034-b3c8-892906a5cd5a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:02.596734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf162d0-7869-4034-b3c8-892906a5cd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:02.671628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6bf162d0-7869-4034-b3c8-892906a5cd5a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:02.672878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6bf162d0-7869-4034-b3c8-892906a5cd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:02.778441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:02.783733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 674/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:31:03.060894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c996a-6ff5-4d1b-addd-34c533db19ef None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:03.062474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b86c996a-6ff5-4d1b-addd-34c533db19ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:03.177648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c996a-6ff5-4d1b-addd-34c533db19ef None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:03.224921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:03.251951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8222f-650d-4bda-9430-75b477f9048d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:03.252570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8222f-650d-4bda-9430-75b477f9048d service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:03.252847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8222f-650d-4bda-9430-75b477f9048d service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:03.253404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8222f-650d-4bda-9430-75b477f9048d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:03.255084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-aac8222f-650d-4bda-9430-75b477f9048d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:03.372999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8222f-650d-4bda-9430-75b477f9048d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:03.374477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8222f-650d-4bda-9430-75b477f9048d service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:03.394329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 675/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:08.286297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcc11d-6130-47aa-a921-2fe44132fb51 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:08.286843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcc11d-6130-47aa-a921-2fe44132fb51 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:08.287075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcc11d-6130-47aa-a921-2fe44132fb51 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:08.292233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 675/1350] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:31:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:08.296898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8c84-9461-4042-995e-a62a00250d3c None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:08.296898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8c84-9461-4042-995e-a62a00250d3c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:08.296898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8c84-9461-4042-995e-a62a00250d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:08.450634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-122a8c84-9461-4042-995e-a62a00250d3c None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:08.451862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-122a8c84-9461-4042-995e-a62a00250d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:08.474499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:08.590451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 676/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:31:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:31:12.786249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1870d1-9018-4b0f-b543-b5b3faeac53a None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:12.786807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1870d1-9018-4b0f-b543-b5b3faeac53a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:12.787045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1870d1-9018-4b0f-b543-b5b3faeac53a None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:12.788542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 676/1352] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:12.793697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b938da-188c-4b31-afb2-50f6dcd9db9e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:12.794540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b938da-188c-4b31-afb2-50f6dcd9db9e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:12.795022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b938da-188c-4b31-afb2-50f6dcd9db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:12.875794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-96b938da-188c-4b31-afb2-50f6dcd9db9e None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:12.877383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-96b938da-188c-4b31-afb2-50f6dcd9db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:12.961739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:13.008477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 677/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:31:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:14.211365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871d9a6-e0aa-4377-a507-34b2a28129b2 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:14.211961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871d9a6-e0aa-4377-a507-34b2a28129b2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:14.212117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871d9a6-e0aa-4377-a507-34b2a28129b2 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:14.213484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 677/1354] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:14.219148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1951-c2b9-49b8-8045-9dfb8427cb86 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:14.219612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1951-c2b9-49b8-8045-9dfb8427cb86 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:14.219835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1951-c2b9-49b8-8045-9dfb8427cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:14.294744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e30d1951-c2b9-49b8-8045-9dfb8427cb86 None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:14.296260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e30d1951-c2b9-49b8-8045-9dfb8427cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:14.406951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 678/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:31:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:31:17.870251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cd821-0a20-417b-8492-626ccdbb7612 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:17.870836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cd821-0a20-417b-8492-626ccdbb7612 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:17.870836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cd821-0a20-417b-8492-626ccdbb7612 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:17.872131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 678/1356] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:17.877475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f9dcb-1f76-41f9-9153-037c3d646e2d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:17.878016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f9dcb-1f76-41f9-9153-037c3d646e2d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:17.878113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f9dcb-1f76-41f9-9153-037c3d646e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:17.914854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:17.963386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-426f9dcb-1f76-41f9-9153-037c3d646e2d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:17.964216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-426f9dcb-1f76-41f9-9153-037c3d646e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:18.088443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 679/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:18.386056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d10fc-b3f3-4dd7-9a98-f9e81d878c6b None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:18.386245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d10fc-b3f3-4dd7-9a98-f9e81d878c6b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:18.386680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d10fc-b3f3-4dd7-9a98-f9e81d878c6b None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:18.388478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 679/1358] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 22 05:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:18.397302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383e297-3f7e-4486-9c9e-f961273cf6b6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:18.397672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383e297-3f7e-4486-9c9e-f961273cf6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:18.398034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383e297-3f7e-4486-9c9e-f961273cf6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:18.495932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d383e297-3f7e-4486-9c9e-f961273cf6b6 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:18.497251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d383e297-3f7e-4486-9c9e-f961273cf6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:18.547699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:18.641099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d10fc-b3f3-4dd7-9a98-f9e81d878c6b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:18.642997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-129d10fc-b3f3-4dd7-9a98-f9e81d878c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:18.643897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 680/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 22 05:31:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:31:18.800598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d10fc-b3f3-4dd7-9a98-f9e81d878c6b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:18.890158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:18.934368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba237d-e455-4bc5-a16c-4924000e12d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:18.935140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba237d-e455-4bc5-a16c-4924000e12d1 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:18.935518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba237d-e455-4bc5-a16c-4924000e12d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:18.935904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba237d-e455-4bc5-a16c-4924000e12d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:18.937611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c9ba237d-e455-4bc5-a16c-4924000e12d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:19.031533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba237d-e455-4bc5-a16c-4924000e12d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:19.033472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba237d-e455-4bc5-a16c-4924000e12d1 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:19.054269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 680/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:21.178512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7abc2-afd5-4491-ad76-832e9fc72412 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:21.179092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7abc2-afd5-4491-ad76-832e9fc72412 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:21.179092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7abc2-afd5-4491-ad76-832e9fc72412 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:21.181778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 681/1361] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:21.186015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1486-81ef-48b1-9ef6-b4ea1fa8c6ef None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:21.186524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1486-81ef-48b1-9ef6-b4ea1fa8c6ef None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:21.186620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1486-81ef-48b1-9ef6-b4ea1fa8c6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:21.257164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-aefa1486-81ef-48b1-9ef6-b4ea1fa8c6ef None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:21.258506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aefa1486-81ef-48b1-9ef6-b4ea1fa8c6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:21.298118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:21.366460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 681/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:21.663819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7abc2-afd5-4491-ad76-832e9fc72412 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:21.663819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6cb7abc2-afd5-4491-ad76-832e9fc72412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:21.840016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7abc2-afd5-4491-ad76-832e9fc72412 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:21.857932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:21.980441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f5aa5-7279-4981-bdd0-2521a725f478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:21.981063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f5aa5-7279-4981-bdd0-2521a725f478 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:21.981340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f5aa5-7279-4981-bdd0-2521a725f478 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:21.981613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f5aa5-7279-4981-bdd0-2521a725f478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:21.983240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4c3f5aa5-7279-4981-bdd0-2521a725f478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:22.080455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f5aa5-7279-4981-bdd0-2521a725f478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:22.082154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f5aa5-7279-4981-bdd0-2521a725f478 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:22.101855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 682/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:24.339246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d98a-0f28-4b10-bfd9-c540bfe73ba8 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:24.339875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d98a-0f28-4b10-bfd9-c540bfe73ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:24.339875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d98a-0f28-4b10-bfd9-c540bfe73ba8 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:24.341132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 682/1364] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:24.347462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aa5f5-0ade-497c-9e65-bc1d31403c7a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:24.349279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aa5f5-0ade-497c-9e65-bc1d31403c7a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:24.349725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aa5f5-0ade-497c-9e65-bc1d31403c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:24.366627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:24.421123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-a73aa5f5-0ade-497c-9e65-bc1d31403c7a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:24.422313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a73aa5f5-0ade-497c-9e65-bc1d31403c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:24.525104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 683/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:24.821162 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352d670-f27c-48ac-a8e7-0fcb5489581f None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:24.821410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352d670-f27c-48ac-a8e7-0fcb5489581f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:24.821890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352d670-f27c-48ac-a8e7-0fcb5489581f None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:24.823198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 683/1366] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 22 05:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:31:24.829136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5721c-3155-4cb3-90bc-756d65c85fbe None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:24.829500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5721c-3155-4cb3-90bc-756d65c85fbe None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:24.829837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5721c-3155-4cb3-90bc-756d65c85fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:24.913018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5ae5721c-3155-4cb3-90bc-756d65c85fbe None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:24.914324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5ae5721c-3155-4cb3-90bc-756d65c85fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:24.951353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:25.037707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 684/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 22 05:31:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:25.045681 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352d670-f27c-48ac-a8e7-0fcb5489581f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:25.047186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9352d670-f27c-48ac-a8e7-0fcb5489581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:25.208752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352d670-f27c-48ac-a8e7-0fcb5489581f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:25.231980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:25.330559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a357b6b-ccd3-4d07-976f-b869f3fdd673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:25.331305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a357b6b-ccd3-4d07-976f-b869f3fdd673 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:25.331817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a357b6b-ccd3-4d07-976f-b869f3fdd673 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:25.332092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a357b6b-ccd3-4d07-976f-b869f3fdd673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:25.333687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9a357b6b-ccd3-4d07-976f-b869f3fdd673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:25.422818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a357b6b-ccd3-4d07-976f-b869f3fdd673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:25.425471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a357b6b-ccd3-4d07-976f-b869f3fdd673 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:25.448438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 684/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:26.045607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445c20-13c0-47f5-997a-19e5006e13f9 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:26.046190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445c20-13c0-47f5-997a-19e5006e13f9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:26.046556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445c20-13c0-47f5-997a-19e5006e13f9 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:26.048126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 685/1369] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:26.053698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86209-7411-4e28-acce-0ae01328d300 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:26.053970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86209-7411-4e28-acce-0ae01328d300 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:26.054265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86209-7411-4e28-acce-0ae01328d300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:26.087992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:26.125482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-8fd86209-7411-4e28-acce-0ae01328d300 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:26.126679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8fd86209-7411-4e28-acce-0ae01328d300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:26.231619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 685/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:26.784750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b25c5-1115-4bde-93b2-90dbd6a2ea27 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:26.785041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b25c5-1115-4bde-93b2-90dbd6a2ea27 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:26.785315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b25c5-1115-4bde-93b2-90dbd6a2ea27 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:26.786694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 686/1371] 10.222.0.18 () {54 vars in 964 bytes} [Mon Aug 22 05:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:26.792087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffd85d-f01f-41c6-b7a9-ad2a387148a5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:26.792569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffd85d-f01f-41c6-b7a9-ad2a387148a5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:26.792821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffd85d-f01f-41c6-b7a9-ad2a387148a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:26.871992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-01ffd85d-f01f-41c6-b7a9-ad2a387148a5 None None] MFA Rules not processed for user `3fcf79bef8d542d9adbfce89931032c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:26.873209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-01ffd85d-f01f-41c6-b7a9-ad2a387148a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:26.923648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:26.955089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 686/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Mon Aug 22 05:31:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:26.963069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b25c5-1115-4bde-93b2-90dbd6a2ea27 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:26.964746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7e9b25c5-1115-4bde-93b2-90dbd6a2ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:27.080455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b25c5-1115-4bde-93b2-90dbd6a2ea27 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:27.125027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:27.163630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24a595-182e-4631-97d1-7fef3a7a2431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:27.164381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24a595-182e-4631-97d1-7fef3a7a2431 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:27.164661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24a595-182e-4631-97d1-7fef3a7a2431 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:27.164931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24a595-182e-4631-97d1-7fef3a7a2431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:27.166940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3f24a595-182e-4631-97d1-7fef3a7a2431 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:27.315708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24a595-182e-4631-97d1-7fef3a7a2431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:27.317322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24a595-182e-4631-97d1-7fef3a7a2431 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:27.342028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 687/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:31:27.484129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12253313-5703-428f-88e2-38781e5b00fd None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:27.484129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12253313-5703-428f-88e2-38781e5b00fd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:27.484129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12253313-5703-428f-88e2-38781e5b00fd None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:27.486472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 687/1374] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:27.492124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012fbd6-fc27-4e18-a0f9-4ac25e2cc547 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:27.492412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012fbd6-fc27-4e18-a0f9-4ac25e2cc547 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:27.492687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012fbd6-fc27-4e18-a0f9-4ac25e2cc547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:27.585300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-0012fbd6-fc27-4e18-a0f9-4ac25e2cc547 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:27.586514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0012fbd6-fc27-4e18-a0f9-4ac25e2cc547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:27.607635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:27.684800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 688/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:27.970013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12253313-5703-428f-88e2-38781e5b00fd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:27.971815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-12253313-5703-428f-88e2-38781e5b00fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:28.125606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12253313-5703-428f-88e2-38781e5b00fd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:28.176170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:28.238847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8436ff-de16-4893-88f4-b82db6da655f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:28.239450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8436ff-de16-4893-88f4-b82db6da655f service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:28.239756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8436ff-de16-4893-88f4-b82db6da655f service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:28.240020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8436ff-de16-4893-88f4-b82db6da655f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:28.241728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9a8436ff-de16-4893-88f4-b82db6da655f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:28.331293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8436ff-de16-4893-88f4-b82db6da655f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:28.333191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8436ff-de16-4893-88f4-b82db6da655f service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:28.351993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 688/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:30.284310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695652c-6c5c-4af5-8d7d-2c0e00494b2a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:30.284869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695652c-6c5c-4af5-8d7d-2c0e00494b2a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:30.284869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695652c-6c5c-4af5-8d7d-2c0e00494b2a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:30.286191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 689/1377] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:30.291721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427f698-211b-496c-a75d-a73e8acc48c7 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:30.292179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427f698-211b-496c-a75d-a73e8acc48c7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:30.292285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427f698-211b-496c-a75d-a73e8acc48c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:30.370264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a427f698-211b-496c-a75d-a73e8acc48c7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:30.371631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a427f698-211b-496c-a75d-a73e8acc48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:30.381995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:30.482897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 689/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:30.767319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695652c-6c5c-4af5-8d7d-2c0e00494b2a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:30.768897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2695652c-6c5c-4af5-8d7d-2c0e00494b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:30.967975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695652c-6c5c-4af5-8d7d-2c0e00494b2a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:30.972254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:31.088874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c705c-bff0-48d7-84fc-18ad6b4cfcb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:31.089620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c705c-bff0-48d7-84fc-18ad6b4cfcb1 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:31.089992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c705c-bff0-48d7-84fc-18ad6b4cfcb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:31.090352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c705c-bff0-48d7-84fc-18ad6b4cfcb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:31.092255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-177c705c-bff0-48d7-84fc-18ad6b4cfcb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:31.187777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c705c-bff0-48d7-84fc-18ad6b4cfcb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:31.189576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c705c-bff0-48d7-84fc-18ad6b4cfcb1 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:31.208584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 690/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:33.176761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56861-654d-4216-b4bf-affcad760a64 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:33.177375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56861-654d-4216-b4bf-affcad760a64 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:33.177375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56861-654d-4216-b4bf-affcad760a64 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:33.178921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 690/1380] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:33.184372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8c035-b5d9-4a05-a78e-a5603ebba30a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:33.184831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8c035-b5d9-4a05-a78e-a5603ebba30a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:33.184930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8c035-b5d9-4a05-a78e-a5603ebba30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:33.196856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:33.261177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-37a8c035-b5d9-4a05-a78e-a5603ebba30a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:33.262407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-37a8c035-b5d9-4a05-a78e-a5603ebba30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:33.367627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 691/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:33.654411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e56861-654d-4216-b4bf-affcad760a64 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:33.656148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a7e56861-654d-4216-b4bf-affcad760a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:33.818312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:33.832700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e56861-654d-4216-b4bf-affcad760a64 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:33.972771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43c0d5-acbe-49b6-8e84-70b82987b2c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:33.973560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c0d5-acbe-49b6-8e84-70b82987b2c9 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:33.973843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c0d5-acbe-49b6-8e84-70b82987b2c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:33.974120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c0d5-acbe-49b6-8e84-70b82987b2c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:33.975759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1a43c0d5-acbe-49b6-8e84-70b82987b2c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:34.071346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c0d5-acbe-49b6-8e84-70b82987b2c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:34.072882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c0d5-acbe-49b6-8e84-70b82987b2c9 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:34.084461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:34.094810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 691/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:36.253684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b80996-baab-4dbe-9853-c31bccd6ebf9 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:36.254279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b80996-baab-4dbe-9853-c31bccd6ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:36.254395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b80996-baab-4dbe-9853-c31bccd6ebf9 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:36.257296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 692/1383] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:36.263135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6491a-029f-4c82-a312-1c237a79ebd5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:36.263831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6491a-029f-4c82-a312-1c237a79ebd5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:36.263831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6491a-029f-4c82-a312-1c237a79ebd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:36.342326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0cc6491a-029f-4c82-a312-1c237a79ebd5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:36.343770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0cc6491a-029f-4c82-a312-1c237a79ebd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:36.450653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 692/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:36.783038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b80996-baab-4dbe-9853-c31bccd6ebf9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:36.784770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-95b80996-baab-4dbe-9853-c31bccd6ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:36.895392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:36.970751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b80996-baab-4dbe-9853-c31bccd6ebf9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:37.096151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acad3d-0792-468e-8b38-c744aa0cb64b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:37.096879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acad3d-0792-468e-8b38-c744aa0cb64b service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:37.097176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acad3d-0792-468e-8b38-c744aa0cb64b service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:37.097547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acad3d-0792-468e-8b38-c744aa0cb64b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:37.099484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f7acad3d-0792-468e-8b38-c744aa0cb64b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:37.160165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:37.210297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acad3d-0792-468e-8b38-c744aa0cb64b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:37.212591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acad3d-0792-468e-8b38-c744aa0cb64b service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:37.236868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 693/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:39.312132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dcb64-4ea7-4d33-86e5-28d39d08d830 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:39.313052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dcb64-4ea7-4d33-86e5-28d39d08d830 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:39.313052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dcb64-4ea7-4d33-86e5-28d39d08d830 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:39.314634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 693/1386] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:39.321591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab6ec6-110e-4dc6-a22c-1bf22c84c9a8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:39.322281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab6ec6-110e-4dc6-a22c-1bf22c84c9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:39.322281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab6ec6-110e-4dc6-a22c-1bf22c84c9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:39.419597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bbab6ec6-110e-4dc6-a22c-1bf22c84c9a8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:39.420959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bbab6ec6-110e-4dc6-a22c-1bf22c84c9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:39.551430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 694/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:39.874021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dcb64-4ea7-4d33-86e5-28d39d08d830 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:39.874021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-522dcb64-4ea7-4d33-86e5-28d39d08d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:39.977623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:40.220267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dcb64-4ea7-4d33-86e5-28d39d08d830 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:40.356013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d98494-da0f-41bc-be87-817fbe1fa637 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:40.356744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98494-da0f-41bc-be87-817fbe1fa637 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:40.356965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98494-da0f-41bc-be87-817fbe1fa637 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:40.357173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98494-da0f-41bc-be87-817fbe1fa637 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:40.358805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e5d98494-da0f-41bc-be87-817fbe1fa637 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:40.406686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:40.468632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d98494-da0f-41bc-be87-817fbe1fa637 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:40.471416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d98494-da0f-41bc-be87-817fbe1fa637 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:40.491990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 694/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:31:42.336779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e7624-1fe2-45f1-87c9-52ca42c28a18 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:42.337294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e7624-1fe2-45f1-87c9-52ca42c28a18 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:42.337294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e7624-1fe2-45f1-87c9-52ca42c28a18 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:42.338633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 695/1389] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:42.344983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cea68-062a-4a1c-a52f-b32536bd747a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:42.345473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cea68-062a-4a1c-a52f-b32536bd747a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:42.345473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cea68-062a-4a1c-a52f-b32536bd747a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:42.412369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-723cea68-062a-4a1c-a52f-b32536bd747a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:42.413575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-723cea68-062a-4a1c-a52f-b32536bd747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:42.469965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07718e-dd55-4fa3-b573-680045df37ed None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:42.470214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07718e-dd55-4fa3-b573-680045df37ed None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:42.470497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07718e-dd55-4fa3-b573-680045df37ed None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:42.472156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 696/1390] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:31:42.477312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e65f7-aee8-4ce8-9560-e1efa56060e0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:42.477585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e65f7-aee8-4ce8-9560-e1efa56060e0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:42.477836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e65f7-aee8-4ce8-9560-e1efa56060e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:42.512688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:42.515198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 695/1391] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:42.548531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e24e65f7-aee8-4ce8-9560-e1efa56060e0 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:42.549775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e24e65f7-aee8-4ce8-9560-e1efa56060e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:42.647347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 697/1392] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:31:42.811602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc185-ff83-4693-bd59-642b7697a213 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:42.811918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc185-ff83-4693-bd59-642b7697a213 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:42.812189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc185-ff83-4693-bd59-642b7697a213 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:42.813541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 696/1393] 10.222.0.18 () {54 vars in 967 bytes} [Mon Aug 22 05:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:42.820276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3f64-0c78-4dc5-a10d-aad99f15edd7 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:42.820532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3f64-0c78-4dc5-a10d-aad99f15edd7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:42.820936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3f64-0c78-4dc5-a10d-aad99f15edd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:42.901181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-45cb3f64-0c78-4dc5-a10d-aad99f15edd7 None None] MFA Rules not processed for user `69d9ee15b5fa43248fbdca69c3c91a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:42.902352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-45cb3f64-0c78-4dc5-a10d-aad99f15edd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:42.935975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc185-ff83-4693-bd59-642b7697a213 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:42.937506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-45dbc185-ff83-4693-bd59-642b7697a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:42.951076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:42.954314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:42.984924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 698/1394] 10.222.0.18 () {58 vars in 1064 bytes} [Mon Aug 22 05:31:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:42.993007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb3f64-0c78-4dc5-a10d-aad99f15edd7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:43.000539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb3f64-0c78-4dc5-a10d-aad99f15edd7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:43.075876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a244b1-b0ec-4ce7-8558-92ef3e0ffd36 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:43.076484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a244b1-b0ec-4ce7-8558-92ef3e0ffd36 service glance] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:43.076764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a244b1-b0ec-4ce7-8558-92ef3e0ffd36 service glance] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:43.077034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a244b1-b0ec-4ce7-8558-92ef3e0ffd36 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:43.078631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-81a244b1-b0ec-4ce7-8558-92ef3e0ffd36 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:43.087127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc185-ff83-4693-bd59-642b7697a213 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:43.177747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:43.192578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a244b1-b0ec-4ce7-8558-92ef3e0ffd36 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:43.194058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a244b1-b0ec-4ce7-8558-92ef3e0ffd36 service glance] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:43.205140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33f635-55cd-4851-b1d6-9693beb1f4fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:43.205780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f635-55cd-4851-b1d6-9693beb1f4fc service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:43.206052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f635-55cd-4851-b1d6-9693beb1f4fc service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:43.206318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f635-55cd-4851-b1d6-9693beb1f4fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:43.207950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0f33f635-55cd-4851-b1d6-9693beb1f4fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:43.208627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:43.213926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 699/1395] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:43.297008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33f635-55cd-4851-b1d6-9693beb1f4fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:43.298574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33f635-55cd-4851-b1d6-9693beb1f4fc service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:43.317159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 697/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:31:43.978249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd689-2a40-4fa7-a24f-d0c84e3fb747 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:43.978722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd689-2a40-4fa7-a24f-d0c84e3fb747 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:43.978822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd689-2a40-4fa7-a24f-d0c84e3fb747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:44.048724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f2bcd689-2a40-4fa7-a24f-d0c84e3fb747 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:44.050037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f2bcd689-2a40-4fa7-a24f-d0c84e3fb747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:44.139249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 700/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:31:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:44.186401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879081a6-2a7d-4955-923f-1e831623b84e None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:44.186938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879081a6-2a7d-4955-923f-1e831623b84e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:44.187063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879081a6-2a7d-4955-923f-1e831623b84e None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:44.188723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 698/1398] 10.222.0.18 () {54 vars in 943 bytes} [Mon Aug 22 05:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:44.195913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca598af-340f-4a98-a30b-82e2837aaee3 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:44.196201 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca598af-340f-4a98-a30b-82e2837aaee3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:44.196494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca598af-340f-4a98-a30b-82e2837aaee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:44.275287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-aca598af-340f-4a98-a30b-82e2837aaee3 None None] MFA Rules not processed for user `b63e2db3e07c4471be6928ff31334816`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:44.276516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-aca598af-340f-4a98-a30b-82e2837aaee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:44.306804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:44.385580 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 701/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Mon Aug 22 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:44.395606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879081a6-2a7d-4955-923f-1e831623b84e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:44.397180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-879081a6-2a7d-4955-923f-1e831623b84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:44.495977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:44.538188 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879081a6-2a7d-4955-923f-1e831623b84e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:44.647881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb7551-2242-4fba-a1d0-a5ff53666939 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:44.648477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb7551-2242-4fba-a1d0-a5ff53666939 service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:44.648746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb7551-2242-4fba-a1d0-a5ff53666939 service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:44.649009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb7551-2242-4fba-a1d0-a5ff53666939 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:44.650618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-2bdb7551-2242-4fba-a1d0-a5ff53666939 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:44.739526 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb7551-2242-4fba-a1d0-a5ff53666939 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:44.741201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb7551-2242-4fba-a1d0-a5ff53666939 service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:44.744690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:44.750711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 699/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:31:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:31:45.267784 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99c549-4c65-4690-a599-4dfec25e69bd None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:45.268423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99c549-4c65-4690-a599-4dfec25e69bd None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:45.268423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99c549-4c65-4690-a599-4dfec25e69bd None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:45.269739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 702/1401] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:45.275135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3015533-b00c-4910-b526-b4cd36c05be4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:45.275625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3015533-b00c-4910-b526-b4cd36c05be4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:45.275769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3015533-b00c-4910-b526-b4cd36c05be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:45.347900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f3015533-b00c-4910-b526-b4cd36c05be4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:45.349157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f3015533-b00c-4910-b526-b4cd36c05be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:45.455752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 700/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:45.749626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99c549-4c65-4690-a599-4dfec25e69bd None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:45.751175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ea99c549-4c65-4690-a599-4dfec25e69bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:45.920329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99c549-4c65-4690-a599-4dfec25e69bd None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:45.926504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:46.052221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c7016-a5bf-4875-85e1-e4b3a7fe9191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:46.053493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c7016-a5bf-4875-85e1-e4b3a7fe9191 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:46.053882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c7016-a5bf-4875-85e1-e4b3a7fe9191 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:46.054150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c7016-a5bf-4875-85e1-e4b3a7fe9191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:46.057267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ee5c7016-a5bf-4875-85e1-e4b3a7fe9191 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:46.155760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c7016-a5bf-4875-85e1-e4b3a7fe9191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:46.157404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c7016-a5bf-4875-85e1-e4b3a7fe9191 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:46.176859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 703/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:48.232111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc668-a2dd-4757-9321-06857ae08c7b None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:48.232737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc668-a2dd-4757-9321-06857ae08c7b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:48.232737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc668-a2dd-4757-9321-06857ae08c7b None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:48.233943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 701/1404] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:48.239415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f138-0763-4051-804d-c079d8dbacc9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:48.239877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f138-0763-4051-804d-c079d8dbacc9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:48.240032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f138-0763-4051-804d-c079d8dbacc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:48.253142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:48.318804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-fdb7f138-0763-4051-804d-c079d8dbacc9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:48.320213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fdb7f138-0763-4051-804d-c079d8dbacc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:48.438644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 704/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:48.741195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdc668-a2dd-4757-9321-06857ae08c7b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:48.742773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7cbdc668-a2dd-4757-9321-06857ae08c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:48.826718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:48.909967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdc668-a2dd-4757-9321-06857ae08c7b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:49.040605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3c30a-9ac2-4eb6-9c39-f1f25831a02d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:49.041227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3c30a-9ac2-4eb6-9c39-f1f25831a02d service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:49.041502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3c30a-9ac2-4eb6-9c39-f1f25831a02d service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:49.041775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3c30a-9ac2-4eb6-9c39-f1f25831a02d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:49.043377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bcd3c30a-9ac2-4eb6-9c39-f1f25831a02d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:49.090180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:49.149044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3c30a-9ac2-4eb6-9c39-f1f25831a02d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:49.150987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3c30a-9ac2-4eb6-9c39-f1f25831a02d service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:49.173417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 702/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:31:51.172917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb902-e2e7-469a-a1f4-262318fd29b3 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:51.173572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb902-e2e7-469a-a1f4-262318fd29b3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:51.173572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb902-e2e7-469a-a1f4-262318fd29b3 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:51.177475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 705/1407] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:31:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:31:51.180637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e6a54-0ae5-4a02-ba88-03307edd99f4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:51.181061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e6a54-0ae5-4a02-ba88-03307edd99f4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:51.181166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e6a54-0ae5-4a02-ba88-03307edd99f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:51.258308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-b10e6a54-0ae5-4a02-ba88-03307edd99f4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:31:51.259560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b10e6a54-0ae5-4a02-ba88-03307edd99f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:51.399664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 703/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:31:51.704266 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb902-e2e7-469a-a1f4-262318fd29b3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:31:51.706181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-648bb902-e2e7-469a-a1f4-262318fd29b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:51.803266 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:51.874976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb902-e2e7-469a-a1f4-262318fd29b3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:31:51.998425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e593-7218-4737-be03-5d7fb6a742de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:31:51.998945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e593-7218-4737-be03-5d7fb6a742de service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:31:51.999223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e593-7218-4737-be03-5d7fb6a742de service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:31:51.999491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e593-7218-4737-be03-5d7fb6a742de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:31:52.001133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-51e7e593-7218-4737-be03-5d7fb6a742de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:31:52.052361 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:31:52.098687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7e593-7218-4737-be03-5d7fb6a742de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:31:52.100793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7e593-7218-4737-be03-5d7fb6a742de service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:31:52.122121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 706/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:01.395924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=93 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:02.119500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=24 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:07.080024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6412-9d6e-4232-99d9-cf490d51fec6 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:07.080766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6412-9d6e-4232-99d9-cf490d51fec6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:07.080766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6412-9d6e-4232-99d9-cf490d51fec6 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:07.082056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 704/1410] 10.222.0.18 () {54 vars in 918 bytes} [Mon Aug 22 05:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:07.087454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc03366-b56b-4f38-a69c-a8edb7789e13 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:07.087931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc03366-b56b-4f38-a69c-a8edb7789e13 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:07.088033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc03366-b56b-4f38-a69c-a8edb7789e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:07.174587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-afc03366-b56b-4f38-a69c-a8edb7789e13 None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:07.175771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-afc03366-b56b-4f38-a69c-a8edb7789e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:07.288906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 707/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Aug 22 05:32:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:07.303796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78c2b-7df7-4818-8f91-3a5e3db978eb None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:07.304084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78c2b-7df7-4818-8f91-3a5e3db978eb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:07.304358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78c2b-7df7-4818-8f91-3a5e3db978eb None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:07.305700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 705/1412] 10.222.0.18 () {54 vars in 918 bytes} [Mon Aug 22 05:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:07.310539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8006a9-4ed6-456d-af20-c676469b5a5b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:07.310806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8006a9-4ed6-456d-af20-c676469b5a5b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:07.311079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8006a9-4ed6-456d-af20-c676469b5a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:07.385958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:07.388647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-ba8006a9-4ed6-456d-af20-c676469b5a5b None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:07.390118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ba8006a9-4ed6-456d-af20-c676469b5a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:07.510283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 708/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Aug 22 05:32:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:32:09.919623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04892139-5f73-4236-adf4-b661740bc5ac None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:09.920147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04892139-5f73-4236-adf4-b661740bc5ac None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:09.920147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04892139-5f73-4236-adf4-b661740bc5ac None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:09.921465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:09.927029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f06a-57f4-48ed-81f2-b982df4ece2d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:09.927527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f06a-57f4-48ed-81f2-b982df4ece2d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:09.927755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f06a-57f4-48ed-81f2-b982df4ece2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:10.004917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4249f06a-57f4-48ed-81f2-b982df4ece2d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:10.006193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4249f06a-57f4-48ed-81f2-b982df4ece2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:10.062553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:10.106381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:10.409200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04892139-5f73-4236-adf4-b661740bc5ac None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:10.410916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-04892139-5f73-4236-adf4-b661740bc5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:10.564725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04892139-5f73-4236-adf4-b661740bc5ac None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:10.642149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:10.686634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dd4b9-5fb3-4afd-91ed-a17c9ce7eeca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:10.687178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd4b9-5fb3-4afd-91ed-a17c9ce7eeca service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:10.687458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd4b9-5fb3-4afd-91ed-a17c9ce7eeca service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:10.687762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dd4b9-5fb3-4afd-91ed-a17c9ce7eeca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:10.689386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-053dd4b9-5fb3-4afd-91ed-a17c9ce7eeca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:10.782205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dd4b9-5fb3-4afd-91ed-a17c9ce7eeca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:10.784457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dd4b9-5fb3-4afd-91ed-a17c9ce7eeca service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:10.805090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 707/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:11.718572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c952ea-67ba-4ff3-b4e3-a6c1e1128289 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:11.719110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c952ea-67ba-4ff3-b4e3-a6c1e1128289 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:11.719652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c952ea-67ba-4ff3-b4e3-a6c1e1128289 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:11.721085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:11.726523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f057-5c16-4965-ab62-d76b3d33e1ba None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:11.726993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f057-5c16-4965-ab62-d76b3d33e1ba None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:11.727099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f057-5c16-4965-ab62-d76b3d33e1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:11.807939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-cc73f057-5c16-4965-ab62-d76b3d33e1ba None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:11.809165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cc73f057-5c16-4965-ab62-d76b3d33e1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:11.837051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:11.919920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:12.076460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c952ea-67ba-4ff3-b4e3-a6c1e1128289 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:12.077979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-63c952ea-67ba-4ff3-b4e3-a6c1e1128289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:12.228181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c952ea-67ba-4ff3-b4e3-a6c1e1128289 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:12.270330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:12.347396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdb547-6490-4624-bdac-e3629e503a94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:12.347978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdb547-6490-4624-bdac-e3629e503a94 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:12.348246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdb547-6490-4624-bdac-e3629e503a94 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:12.348509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdb547-6490-4624-bdac-e3629e503a94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:12.350113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-92cdb547-6490-4624-bdac-e3629e503a94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:12.443211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdb547-6490-4624-bdac-e3629e503a94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:12.445073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdb547-6490-4624-bdac-e3629e503a94 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:12.463920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 711/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:13.363080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678558dd-363f-46dd-8a7c-08d72aefd95c None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:13.363631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678558dd-363f-46dd-8a7c-08d72aefd95c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:13.363631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678558dd-363f-46dd-8a7c-08d72aefd95c None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:13.364894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:13.370126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838ad6-2dcc-4f51-a3e6-8d63ea378888 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:13.370579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838ad6-2dcc-4f51-a3e6-8d63ea378888 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:13.370682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838ad6-2dcc-4f51-a3e6-8d63ea378888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:13.429623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:13.455972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-93838ad6-2dcc-4f51-a3e6-8d63ea378888 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:13.458380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-93838ad6-2dcc-4f51-a3e6-8d63ea378888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:13.558459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:13.713452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678558dd-363f-46dd-8a7c-08d72aefd95c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:13.715100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-678558dd-363f-46dd-8a7c-08d72aefd95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:13.865289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678558dd-363f-46dd-8a7c-08d72aefd95c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:13.877622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:13.987052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cb0b4-4734-4c36-8910-10edfee98631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:13.987696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb0b4-4734-4c36-8910-10edfee98631 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:13.987976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb0b4-4734-4c36-8910-10edfee98631 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:13.988247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb0b4-4734-4c36-8910-10edfee98631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:13.989852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-220cb0b4-4734-4c36-8910-10edfee98631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:14.080375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cb0b4-4734-4c36-8910-10edfee98631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:14.082257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cb0b4-4734-4c36-8910-10edfee98631 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:14.101878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 710/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:14.964078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235947b-302d-44bf-a8af-ca073be81cba None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:14.964608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235947b-302d-44bf-a8af-ca073be81cba None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:14.964608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235947b-302d-44bf-a8af-ca073be81cba None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:14.966003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:14.971773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845442d6-d61c-44f9-936e-2fcb4185ea63 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:14.972244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845442d6-d61c-44f9-936e-2fcb4185ea63 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:14.972353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845442d6-d61c-44f9-936e-2fcb4185ea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:14.992753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:15.055327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-845442d6-d61c-44f9-936e-2fcb4185ea63 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:15.056557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-845442d6-d61c-44f9-936e-2fcb4185ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:15.167339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:32:15.320585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235947b-302d-44bf-a8af-ca073be81cba None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:15.322110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0235947b-302d-44bf-a8af-ca073be81cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:15.440738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:15.478710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235947b-302d-44bf-a8af-ca073be81cba None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:15.598934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31978a71-2f8d-4394-ae61-ce19304583fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:15.599554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31978a71-2f8d-4394-ae61-ce19304583fd service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:15.599865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31978a71-2f8d-4394-ae61-ce19304583fd service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:15.600134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31978a71-2f8d-4394-ae61-ce19304583fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:15.601765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-31978a71-2f8d-4394-ae61-ce19304583fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:15.691571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:15.707425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31978a71-2f8d-4394-ae61-ce19304583fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:15.709086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31978a71-2f8d-4394-ae61-ce19304583fd service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:15.732017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 714/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:16.602287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a30aa-9df5-440a-80a7-9ffe0adb7830 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:16.602892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a30aa-9df5-440a-80a7-9ffe0adb7830 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:16.602892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a30aa-9df5-440a-80a7-9ffe0adb7830 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:16.604228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:16.609447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caad67-b9f9-48ec-8b11-64235a389268 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:16.609899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caad67-b9f9-48ec-8b11-64235a389268 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:16.610002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caad67-b9f9-48ec-8b11-64235a389268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:16.695888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-91caad67-b9f9-48ec-8b11-64235a389268 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:16.697092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-91caad67-b9f9-48ec-8b11-64235a389268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:16.797771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:16.950814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a30aa-9df5-440a-80a7-9ffe0adb7830 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:16.952370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c70a30aa-9df5-440a-80a7-9ffe0adb7830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:17.049318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:17.123987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a30aa-9df5-440a-80a7-9ffe0adb7830 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:17.259071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5274-ec3b-4d30-a48f-a8b2190cc845 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:17.259620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5274-ec3b-4d30-a48f-a8b2190cc845 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:17.259916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5274-ec3b-4d30-a48f-a8b2190cc845 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:17.260186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5274-ec3b-4d30-a48f-a8b2190cc845 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:17.261806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0d8a5274-ec3b-4d30-a48f-a8b2190cc845 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:17.313079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:17.355601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5274-ec3b-4d30-a48f-a8b2190cc845 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:17.357229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5274-ec3b-4d30-a48f-a8b2190cc845 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:17.375610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 713/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:18.175928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf917-10b7-48f7-a9b7-6f1b3b7885d1 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:18.176832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf917-10b7-48f7-a9b7-6f1b3b7885d1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:18.176832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaf917-10b7-48f7-a9b7-6f1b3b7885d1 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:18.178743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:18.187357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d918d0-6437-4d9d-865b-bb090490ea08 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:18.188176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d918d0-6437-4d9d-865b-bb090490ea08 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:18.188345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d918d0-6437-4d9d-865b-bb090490ea08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:18.285468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-18d918d0-6437-4d9d-865b-bb090490ea08 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:18.286689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-18d918d0-6437-4d9d-865b-bb090490ea08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:18.391168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:18.545769 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaf917-10b7-48f7-a9b7-6f1b3b7885d1 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:18.548160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-eafaf917-10b7-48f7-a9b7-6f1b3b7885d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:18.607601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:18.725312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaf917-10b7-48f7-a9b7-6f1b3b7885d1 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:18.858665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41b56-0342-449c-94a7-142b206844b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:18.859187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41b56-0342-449c-94a7-142b206844b0 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:18.859471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41b56-0342-449c-94a7-142b206844b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:18.859772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41b56-0342-449c-94a7-142b206844b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:18.861582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-def41b56-0342-449c-94a7-142b206844b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:18.869132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:18.960871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def41b56-0342-449c-94a7-142b206844b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:18.962492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def41b56-0342-449c-94a7-142b206844b0 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:18.981255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 717/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:19.818481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030b35-0f34-4a6a-85ac-df41d7fd1fae None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:19.819117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030b35-0f34-4a6a-85ac-df41d7fd1fae None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:19.819117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030b35-0f34-4a6a-85ac-df41d7fd1fae None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:19.820429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:19.825716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96364-7be5-444d-a366-26801028a688 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:19.826204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96364-7be5-444d-a366-26801028a688 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:19.826204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96364-7be5-444d-a366-26801028a688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:19.908453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b0c96364-7be5-444d-a366-26801028a688 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:19.909716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b0c96364-7be5-444d-a366-26801028a688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:19.986808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:20.022926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:20.180868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b030b35-0f34-4a6a-85ac-df41d7fd1fae None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:20.182392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9b030b35-0f34-4a6a-85ac-df41d7fd1fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:20.202531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:20.354763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b030b35-0f34-4a6a-85ac-df41d7fd1fae None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:20.448289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:20.482530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767534aa-8e0e-4958-ba12-595287967dee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:20.483381 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767534aa-8e0e-4958-ba12-595287967dee service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:20.483770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767534aa-8e0e-4958-ba12-595287967dee service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:20.484078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767534aa-8e0e-4958-ba12-595287967dee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:20.486590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-767534aa-8e0e-4958-ba12-595287967dee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:20.587520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767534aa-8e0e-4958-ba12-595287967dee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:20.589169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767534aa-8e0e-4958-ba12-595287967dee service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:20.607907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 716/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:21.430697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79cb30-c84c-4cb2-a3b6-b80bb5c17dc7 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:21.431647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79cb30-c84c-4cb2-a3b6-b80bb5c17dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:21.431647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79cb30-c84c-4cb2-a3b6-b80bb5c17dc7 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:21.433717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:21.441714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631f8db-3d11-40bb-acbc-bdea78c068ca None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:21.442145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631f8db-3d11-40bb-acbc-bdea78c068ca None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:21.442249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631f8db-3d11-40bb-acbc-bdea78c068ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:21.523323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e631f8db-3d11-40bb-acbc-bdea78c068ca None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:21.524603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e631f8db-3d11-40bb-acbc-bdea78c068ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:21.568271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:21.635840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:21.788256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79cb30-c84c-4cb2-a3b6-b80bb5c17dc7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:21.790491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7a79cb30-c84c-4cb2-a3b6-b80bb5c17dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:21.968700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79cb30-c84c-4cb2-a3b6-b80bb5c17dc7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:22.036928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:22.099609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095bae-9fb2-4dd5-9830-4ea22762c0e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:22.100191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095bae-9fb2-4dd5-9830-4ea22762c0e3 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:22.100470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095bae-9fb2-4dd5-9830-4ea22762c0e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:22.100741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095bae-9fb2-4dd5-9830-4ea22762c0e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:22.102365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-49095bae-9fb2-4dd5-9830-4ea22762c0e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:22.202097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095bae-9fb2-4dd5-9830-4ea22762c0e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:22.203757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095bae-9fb2-4dd5-9830-4ea22762c0e3 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:22.223272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 720/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:23.031729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184df803-2acb-4041-9e6d-440bbd5fe466 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:23.032368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184df803-2acb-4041-9e6d-440bbd5fe466 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:23.032368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184df803-2acb-4041-9e6d-440bbd5fe466 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:23.033682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:23.038929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee4e54-23fa-4545-96e7-092bb290def9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:23.039375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee4e54-23fa-4545-96e7-092bb290def9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:23.039481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee4e54-23fa-4545-96e7-092bb290def9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:23.126492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-36ee4e54-23fa-4545-96e7-092bb290def9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:23.128084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-36ee4e54-23fa-4545-96e7-092bb290def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:23.130279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:23.239448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:23.392738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184df803-2acb-4041-9e6d-440bbd5fe466 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:23.394337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-184df803-2acb-4041-9e6d-440bbd5fe466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:23.548436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184df803-2acb-4041-9e6d-440bbd5fe466 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:23.583346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:23.673176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11f43a-987c-4894-a9eb-cbd765869e4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:23.673884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11f43a-987c-4894-a9eb-cbd765869e4b service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:23.674393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11f43a-987c-4894-a9eb-cbd765869e4b service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:23.674670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11f43a-987c-4894-a9eb-cbd765869e4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:23.676346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8c11f43a-987c-4894-a9eb-cbd765869e4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:23.771542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11f43a-987c-4894-a9eb-cbd765869e4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:23.773163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11f43a-987c-4894-a9eb-cbd765869e4b service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:23.794360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 719/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:25.628379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f57e-3646-4427-9e17-e6d41e315f8f None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:25.628979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f57e-3646-4427-9e17-e6d41e315f8f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:25.628979 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f57e-3646-4427-9e17-e6d41e315f8f None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:25.630263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:25.636820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb677f-982f-42cf-bec6-8d968f66a06c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:25.637550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb677f-982f-42cf-bec6-8d968f66a06c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:25.637717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb677f-982f-42cf-bec6-8d968f66a06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:25.687558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:25.715233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e0cb677f-982f-42cf-bec6-8d968f66a06c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:25.716482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e0cb677f-982f-42cf-bec6-8d968f66a06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:25.816921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:26.117822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358f57e-3646-4427-9e17-e6d41e315f8f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:26.119386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f358f57e-3646-4427-9e17-e6d41e315f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:26.234752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358f57e-3646-4427-9e17-e6d41e315f8f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:26.268022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:26.316342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3cb16-118f-4153-ad28-48ef52490839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:26.316960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3cb16-118f-4153-ad28-48ef52490839 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:26.317278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3cb16-118f-4153-ad28-48ef52490839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:26.317550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3cb16-118f-4153-ad28-48ef52490839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:26.319385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-76d3cb16-118f-4153-ad28-48ef52490839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:26.450066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3cb16-118f-4153-ad28-48ef52490839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:26.451531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3cb16-118f-4153-ad28-48ef52490839 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:26.473537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 723/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:32:28.546669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20052342-a170-4e7e-9b2a-efd0223f9246 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:28.547289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20052342-a170-4e7e-9b2a-efd0223f9246 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:28.547289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20052342-a170-4e7e-9b2a-efd0223f9246 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:28.548638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:28.554941 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac930d5-68b5-478e-8004-0f01eb930fcb None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:28.555558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac930d5-68b5-478e-8004-0f01eb930fcb None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:28.555694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac930d5-68b5-478e-8004-0f01eb930fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:28.640181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4ac930d5-68b5-478e-8004-0f01eb930fcb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:28.641400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4ac930d5-68b5-478e-8004-0f01eb930fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:28.654467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:28.754899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:29.038785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20052342-a170-4e7e-9b2a-efd0223f9246 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:29.040389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-20052342-a170-4e7e-9b2a-efd0223f9246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:29.161176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20052342-a170-4e7e-9b2a-efd0223f9246 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:29.173705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:29.248564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0f779-1ca7-4bbb-9466-9f7aac023eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:29.249484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0f779-1ca7-4bbb-9466-9f7aac023eba service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:29.249849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0f779-1ca7-4bbb-9466-9f7aac023eba service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:29.250186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0f779-1ca7-4bbb-9466-9f7aac023eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:29.252864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-afb0f779-1ca7-4bbb-9466-9f7aac023eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:29.376792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0f779-1ca7-4bbb-9466-9f7aac023eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:29.378659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0f779-1ca7-4bbb-9466-9f7aac023eba service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:29.398260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 722/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:31.606354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e593e-75a5-45f7-8c75-bce0c180a02d None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:31.606989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e593e-75a5-45f7-8c75-bce0c180a02d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:31.606989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e593e-75a5-45f7-8c75-bce0c180a02d None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:31.608281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:31.614912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803eb72-1f99-4b3d-949c-d673ccc61f1a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:31.615336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803eb72-1f99-4b3d-949c-d673ccc61f1a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:31.615483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803eb72-1f99-4b3d-949c-d673ccc61f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:31.676251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:31.696791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c803eb72-1f99-4b3d-949c-d673ccc61f1a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:31.698074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c803eb72-1f99-4b3d-949c-d673ccc61f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:31.799504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:32.084795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e593e-75a5-45f7-8c75-bce0c180a02d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:32.086338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-636e593e-75a5-45f7-8c75-bce0c180a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:32.194867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e593e-75a5-45f7-8c75-bce0c180a02d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:32.236602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:32.273867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987968ca-90ad-480e-b614-52aa71daf744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:32.274482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987968ca-90ad-480e-b614-52aa71daf744 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:32.274774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987968ca-90ad-480e-b614-52aa71daf744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:32.275039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987968ca-90ad-480e-b614-52aa71daf744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:32.276694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-987968ca-90ad-480e-b614-52aa71daf744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:32.393690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987968ca-90ad-480e-b614-52aa71daf744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:32.395393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987968ca-90ad-480e-b614-52aa71daf744 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:32.414282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 726/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:34.466679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1df4-add3-430d-a6ff-e8835f526cd2 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:34.467282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1df4-add3-430d-a6ff-e8835f526cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:34.467282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1df4-add3-430d-a6ff-e8835f526cd2 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:34.468622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:34.475923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b4c4-18e0-4306-9a80-2628be071094 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:34.476367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b4c4-18e0-4306-9a80-2628be071094 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:34.476482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b4c4-18e0-4306-9a80-2628be071094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:34.550837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7ee6b4c4-18e0-4306-9a80-2628be071094 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:34.552099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7ee6b4c4-18e0-4306-9a80-2628be071094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:34.575449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:34.660694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:32:34.947065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d1df4-add3-430d-a6ff-e8835f526cd2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:34.949601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ba6d1df4-add3-430d-a6ff-e8835f526cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:35.071221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d1df4-add3-430d-a6ff-e8835f526cd2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:35.091246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:35.144547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ac8a5-ed9c-45c9-82be-9721a35726c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:35.145157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ac8a5-ed9c-45c9-82be-9721a35726c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:35.145435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ac8a5-ed9c-45c9-82be-9721a35726c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:35.145689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ac8a5-ed9c-45c9-82be-9721a35726c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:35.147266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-511ac8a5-ed9c-45c9-82be-9721a35726c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:35.274273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ac8a5-ed9c-45c9-82be-9721a35726c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:35.275652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ac8a5-ed9c-45c9-82be-9721a35726c5 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:35.294300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 725/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:37.306606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629070-15ad-4a53-9683-505f5376a9c7 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:37.307161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629070-15ad-4a53-9683-505f5376a9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:37.307161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629070-15ad-4a53-9683-505f5376a9c7 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:37.308403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:37.314900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef4b7-cf5a-4e44-b354-57bb19442bac None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:37.315650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef4b7-cf5a-4e44-b354-57bb19442bac None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:37.315847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef4b7-cf5a-4e44-b354-57bb19442bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:37.382084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:37.387221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-cb1ef4b7-cf5a-4e44-b354-57bb19442bac None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:37.388826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cb1ef4b7-cf5a-4e44-b354-57bb19442bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:37.501974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:37.785271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629070-15ad-4a53-9683-505f5376a9c7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:37.788192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-87629070-15ad-4a53-9683-505f5376a9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:37.927021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629070-15ad-4a53-9683-505f5376a9c7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:37.983382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:38.009042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc505f7a-3cc4-4aa7-bc60-abe1b1d2bbb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:38.009650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc505f7a-3cc4-4aa7-bc60-abe1b1d2bbb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:38.009929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc505f7a-3cc4-4aa7-bc60-abe1b1d2bbb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:38.010198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc505f7a-3cc4-4aa7-bc60-abe1b1d2bbb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:38.011816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-dc505f7a-3cc4-4aa7-bc60-abe1b1d2bbb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:38.134886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc505f7a-3cc4-4aa7-bc60-abe1b1d2bbb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:38.136364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc505f7a-3cc4-4aa7-bc60-abe1b1d2bbb1 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:38.154194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 729/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:40.130006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd62b-f530-45c6-bde8-d05d52bd1b98 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:40.130603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd62b-f530-45c6-bde8-d05d52bd1b98 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:40.130603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd62b-f530-45c6-bde8-d05d52bd1b98 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:40.133095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:40.138687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa66130-26a4-4011-a319-063b828977af None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:40.139175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa66130-26a4-4011-a319-063b828977af None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:40.139175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa66130-26a4-4011-a319-063b828977af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:40.230014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-daa66130-26a4-4011-a319-063b828977af None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:40.232692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-daa66130-26a4-4011-a319-063b828977af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:40.274459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:40.359689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:40.685177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bd62b-f530-45c6-bde8-d05d52bd1b98 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:40.686763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c67bd62b-f530-45c6-bde8-d05d52bd1b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:40.812791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bd62b-f530-45c6-bde8-d05d52bd1b98 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:40.847178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:40.904780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a5b6a-006d-4533-a897-22c33f700b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:40.905401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a5b6a-006d-4533-a897-22c33f700b94 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:40.905670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a5b6a-006d-4533-a897-22c33f700b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:40.905932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a5b6a-006d-4533-a897-22c33f700b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:40.907607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-158a5b6a-006d-4533-a897-22c33f700b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:41.059427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a5b6a-006d-4533-a897-22c33f700b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:41.060822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a5b6a-006d-4533-a897-22c33f700b94 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:41.083919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 728/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:32:43.135355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e33f3-efc0-4a24-855f-50b06039d318 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:43.135840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e33f3-efc0-4a24-855f-50b06039d318 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:43.136123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e33f3-efc0-4a24-855f-50b06039d318 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:43.138247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:43.146929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de670-d297-48f9-bc6f-8b3489a2159b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:43.147421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de670-d297-48f9-bc6f-8b3489a2159b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:43.147523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de670-d297-48f9-bc6f-8b3489a2159b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:43.223366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-1d4de670-d297-48f9-bc6f-8b3489a2159b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:43.225371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1d4de670-d297-48f9-bc6f-8b3489a2159b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:43.236313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:43.333268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:43.633161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e33f3-efc0-4a24-855f-50b06039d318 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:43.635732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-574e33f3-efc0-4a24-855f-50b06039d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:43.750002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e33f3-efc0-4a24-855f-50b06039d318 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:43.812195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:43.838106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87f4d2-d0b4-4b99-8fa0-2a7e54a7ba9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:43.838723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87f4d2-d0b4-4b99-8fa0-2a7e54a7ba9e service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:43.839013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87f4d2-d0b4-4b99-8fa0-2a7e54a7ba9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:43.839283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87f4d2-d0b4-4b99-8fa0-2a7e54a7ba9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:43.840952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cb87f4d2-d0b4-4b99-8fa0-2a7e54a7ba9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:43.954294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87f4d2-d0b4-4b99-8fa0-2a7e54a7ba9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:43.955788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87f4d2-d0b4-4b99-8fa0-2a7e54a7ba9e service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:43.974848 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 732/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:45.926832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa57c-a672-40ef-87e6-dd78ddf78040 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:45.927373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa57c-a672-40ef-87e6-dd78ddf78040 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:45.927373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa57c-a672-40ef-87e6-dd78ddf78040 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:45.928792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:45.934298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0926e-2930-407c-999c-7757287ea567 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:45.934725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0926e-2930-407c-999c-7757287ea567 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:45.934839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0926e-2930-407c-999c-7757287ea567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:46.009858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-efe0926e-2930-407c-999c-7757287ea567 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:46.011089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-efe0926e-2930-407c-999c-7757287ea567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:46.056688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:46.123815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:46.405452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fa57c-a672-40ef-87e6-dd78ddf78040 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:46.406986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9c6fa57c-a672-40ef-87e6-dd78ddf78040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:46.520638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fa57c-a672-40ef-87e6-dd78ddf78040 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:46.559300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:46.606905 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e640b-d7c0-4303-8282-b46dc5f97ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:46.607520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e640b-d7c0-4303-8282-b46dc5f97ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:46.607829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e640b-d7c0-4303-8282-b46dc5f97ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:46.608103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e640b-d7c0-4303-8282-b46dc5f97ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:46.609745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-904e640b-d7c0-4303-8282-b46dc5f97ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:46.723181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e640b-d7c0-4303-8282-b46dc5f97ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:46.724569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e640b-d7c0-4303-8282-b46dc5f97ed5 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:46.743178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 731/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:32:48.698581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abc95f-cd46-495a-9c9a-b63b9d0fde2c None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:48.699527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abc95f-cd46-495a-9c9a-b63b9d0fde2c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:48.699527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abc95f-cd46-495a-9c9a-b63b9d0fde2c None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:48.701647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:48.707850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d158304-346a-4c11-8eb3-d033942ca82e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:48.708311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d158304-346a-4c11-8eb3-d033942ca82e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:48.708417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d158304-346a-4c11-8eb3-d033942ca82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:48.786906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-8d158304-346a-4c11-8eb3-d033942ca82e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:48.788763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8d158304-346a-4c11-8eb3-d033942ca82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:48.812530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:48.904855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:49.194472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abc95f-cd46-495a-9c9a-b63b9d0fde2c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:49.196092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-53abc95f-cd46-495a-9c9a-b63b9d0fde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:49.320062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abc95f-cd46-495a-9c9a-b63b9d0fde2c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:49.406379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:49.414552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95886be-a7ef-47e4-9b49-28d9d0029aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:49.415175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95886be-a7ef-47e4-9b49-28d9d0029aee service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:49.415454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95886be-a7ef-47e4-9b49-28d9d0029aee service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:49.415757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95886be-a7ef-47e4-9b49-28d9d0029aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:49.417389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e95886be-a7ef-47e4-9b49-28d9d0029aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:49.541102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95886be-a7ef-47e4-9b49-28d9d0029aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:49.542542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95886be-a7ef-47e4-9b49-28d9d0029aee service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:49.564336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 735/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:32:51.518405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b379-d985-45b9-943b-4973e399859e None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:51.519293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b379-d985-45b9-943b-4973e399859e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:51.519293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b379-d985-45b9-943b-4973e399859e None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:51.521367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:32:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:32:51.530589 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0ceb-a2d4-4237-bd87-ddd53879b7b8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:51.531050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0ceb-a2d4-4237-bd87-ddd53879b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:51.531188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad0ceb-a2d4-4237-bd87-ddd53879b7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:51.606770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2dad0ceb-a2d4-4237-bd87-ddd53879b7b8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:32:51.608051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2dad0ceb-a2d4-4237-bd87-ddd53879b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:51.668633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:51.719046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:32:52.008669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253b379-d985-45b9-943b-4973e399859e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:32:52.011211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3253b379-d985-45b9-943b-4973e399859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:52.124918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253b379-d985-45b9-943b-4973e399859e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:32:52.178062 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:32:52.210997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57715289-ad57-4786-bc9c-0704c9c2a166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:32:52.211613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57715289-ad57-4786-bc9c-0704c9c2a166 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:32:52.211918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57715289-ad57-4786-bc9c-0704c9c2a166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:32:52.212180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57715289-ad57-4786-bc9c-0704c9c2a166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:32:52.213797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-57715289-ad57-4786-bc9c-0704c9c2a166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:32:52.337676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57715289-ad57-4786-bc9c-0704c9c2a166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:32:52.339137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57715289-ad57-4786-bc9c-0704c9c2a166 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:32:52.359621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 734/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:33:01.717234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=22 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:02.356954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=66 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:11.862018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbcb15-2be3-4210-a318-7f9e61424f13 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:11.862953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbcb15-2be3-4210-a318-7f9e61424f13 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:11.862953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbcb15-2be3-4210-a318-7f9e61424f13 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:11.864860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 737/1471] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:33:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:11.870418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89548-6dd1-4249-b97e-c445ec7961f4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:11.870861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89548-6dd1-4249-b97e-c445ec7961f4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:11.870974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89548-6dd1-4249-b97e-c445ec7961f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:11.941970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-abc89548-6dd1-4249-b97e-c445ec7961f4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:11.943199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-abc89548-6dd1-4249-b97e-c445ec7961f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:12.048136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 735/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:12.350028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b70bb-6bfe-43ac-a8ad-9aff99fba160 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:12.350313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b70bb-6bfe-43ac-a8ad-9aff99fba160 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:12.350585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b70bb-6bfe-43ac-a8ad-9aff99fba160 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:12.351934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 738/1473] 10.222.0.18 () {54 vars in 967 bytes} [Mon Aug 22 05:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:12.357824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecf384-f025-4e5f-b10b-cd3d69fbc32e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:12.358150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecf384-f025-4e5f-b10b-cd3d69fbc32e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:12.358503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecf384-f025-4e5f-b10b-cd3d69fbc32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:12.441835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-45ecf384-f025-4e5f-b10b-cd3d69fbc32e None None] MFA Rules not processed for user `69d9ee15b5fa43248fbdca69c3c91a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:12.443040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-45ecf384-f025-4e5f-b10b-cd3d69fbc32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:12.466274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:12.533065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 736/1474] 10.222.0.18 () {58 vars in 1064 bytes} [Mon Aug 22 05:33:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:12.541021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b70bb-6bfe-43ac-a8ad-9aff99fba160 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:33:12.542537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0a5b70bb-6bfe-43ac-a8ad-9aff99fba160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:12.656380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b70bb-6bfe-43ac-a8ad-9aff99fba160 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:33:12.735377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bef99-519a-438b-94a3-ff26ed13ccb6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:33:12.736048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bef99-519a-438b-94a3-ff26ed13ccb6 service glance] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:12.736318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bef99-519a-438b-94a3-ff26ed13ccb6 service glance] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:12.736582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bef99-519a-438b-94a3-ff26ed13ccb6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:12.738234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-990bef99-519a-438b-94a3-ff26ed13ccb6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:12.794282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:12.852336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bef99-519a-438b-94a3-ff26ed13ccb6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:33:12.853798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bef99-519a-438b-94a3-ff26ed13ccb6 service glance] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:33:12.874739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 739/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:33:14.895051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aea92-f523-4e4d-86c5-6d75b7dba270 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:14.895953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aea92-f523-4e4d-86c5-6d75b7dba270 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:14.895953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aea92-f523-4e4d-86c5-6d75b7dba270 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:14.897051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 737/1476] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:33:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:14.905224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c02a9a-806e-432b-bc8a-b8db9b34d23a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:14.905648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c02a9a-806e-432b-bc8a-b8db9b34d23a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:14.905742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c02a9a-806e-432b-bc8a-b8db9b34d23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:14.977697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-a6c02a9a-806e-432b-bc8a-b8db9b34d23a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:14.978890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a6c02a9a-806e-432b-bc8a-b8db9b34d23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:15.092994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 740/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:15.369186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aea92-f523-4e4d-86c5-6d75b7dba270 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:33:15.371756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-185aea92-f523-4e4d-86c5-6d75b7dba270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:15.527905 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aea92-f523-4e4d-86c5-6d75b7dba270 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:33:15.583357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:15.632182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7429cc-6f5d-426e-a04a-449dc1c1bd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:33:15.633088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7429cc-6f5d-426e-a04a-449dc1c1bd53 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:15.633484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7429cc-6f5d-426e-a04a-449dc1c1bd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:15.633923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7429cc-6f5d-426e-a04a-449dc1c1bd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:15.634904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7429cc-6f5d-426e-a04a-449dc1c1bd53 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:33:15.637707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-ea7429cc-6f5d-426e-a04a-449dc1c1bd53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:33:15.637707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:33:15.637707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:33:15.639514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7429cc-6f5d-426e-a04a-449dc1c1bd53 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:33:15.667621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 738/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 22 05:33:15 2022] GET /identity/v3/projects => generated 2553 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:33:17.626330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989948b-7d2b-489a-80ee-016e31e1f8c8 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:17.627176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989948b-7d2b-489a-80ee-016e31e1f8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:17.627176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989948b-7d2b-489a-80ee-016e31e1f8c8 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:17.628973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 741/1479] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:33:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:17.634895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfad05-efb6-41e2-a4ba-ac80710ae4d3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:17.635328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfad05-efb6-41e2-a4ba-ac80710ae4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:17.635474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfad05-efb6-41e2-a4ba-ac80710ae4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:17.713869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e4bfad05-efb6-41e2-a4ba-ac80710ae4d3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:17.715486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e4bfad05-efb6-41e2-a4ba-ac80710ae4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:17.822329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 739/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:18.139039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989948b-7d2b-489a-80ee-016e31e1f8c8 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:33:18.141609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c989948b-7d2b-489a-80ee-016e31e1f8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:18.153082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:18.297175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989948b-7d2b-489a-80ee-016e31e1f8c8 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:33:18.392752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:18.428248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c416dfb-d4c6-4289-a892-aaf713343df4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:33:18.428819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416dfb-d4c6-4289-a892-aaf713343df4 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:18.429095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416dfb-d4c6-4289-a892-aaf713343df4 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:18.429362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416dfb-d4c6-4289-a892-aaf713343df4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:18.431028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4c416dfb-d4c6-4289-a892-aaf713343df4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:18.528502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c416dfb-d4c6-4289-a892-aaf713343df4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:33:18.530653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c416dfb-d4c6-4289-a892-aaf713343df4 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:33:18.554081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 742/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:33:20.647804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fc05e-bc2c-44e6-8eba-04a3c05808f9 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:20.648418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fc05e-bc2c-44e6-8eba-04a3c05808f9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:20.648418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fc05e-bc2c-44e6-8eba-04a3c05808f9 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:20.649697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 740/1482] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:20.656211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed003c5-c589-4ac8-a152-41712e649343 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:20.656899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed003c5-c589-4ac8-a152-41712e649343 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:20.657044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed003c5-c589-4ac8-a152-41712e649343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:20.731696 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-1ed003c5-c589-4ac8-a152-41712e649343 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:20.733040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1ed003c5-c589-4ac8-a152-41712e649343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:20.783381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:20.842052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 743/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:21.128158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fc05e-bc2c-44e6-8eba-04a3c05808f9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:33:21.130837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-de6fc05e-bc2c-44e6-8eba-04a3c05808f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:21.168192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:21.259067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fc05e-bc2c-44e6-8eba-04a3c05808f9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:33:21.335474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75aaccb-c2b0-409d-b13e-013d6639ccf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:33:21.336122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aaccb-c2b0-409d-b13e-013d6639ccf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:21.336397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aaccb-c2b0-409d-b13e-013d6639ccf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:21.336660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aaccb-c2b0-409d-b13e-013d6639ccf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:21.338276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a75aaccb-c2b0-409d-b13e-013d6639ccf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:21.418165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:21.461558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75aaccb-c2b0-409d-b13e-013d6639ccf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:33:21.463009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75aaccb-c2b0-409d-b13e-013d6639ccf3 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:33:21.482444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 741/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:33:23.474587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74eade-4940-4b60-aae6-31fb56d37996 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:23.475185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74eade-4940-4b60-aae6-31fb56d37996 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:23.475185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74eade-4940-4b60-aae6-31fb56d37996 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:23.476515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 744/1485] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:23.481961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2ee3-2842-4aa0-b6d5-1a6cab06c027 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:23.482396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2ee3-2842-4aa0-b6d5-1a6cab06c027 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:23.482530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2ee3-2842-4aa0-b6d5-1a6cab06c027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:23.552356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-95ed2ee3-2842-4aa0-b6d5-1a6cab06c027 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:23.553578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-95ed2ee3-2842-4aa0-b6d5-1a6cab06c027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:23.655878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 742/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:23.942104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74eade-4940-4b60-aae6-31fb56d37996 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:33:23.943770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fe74eade-4940-4b60-aae6-31fb56d37996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:24.058566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74eade-4940-4b60-aae6-31fb56d37996 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:33:24.132875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:24.134704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95c38-a057-43e9-8c66-317fd54faf54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:33:24.135363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95c38-a057-43e9-8c66-317fd54faf54 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:24.135669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95c38-a057-43e9-8c66-317fd54faf54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:24.135951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95c38-a057-43e9-8c66-317fd54faf54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:24.138274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1fe95c38-a057-43e9-8c66-317fd54faf54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:24.260903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95c38-a057-43e9-8c66-317fd54faf54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:33:24.262362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95c38-a057-43e9-8c66-317fd54faf54 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:33:24.281240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 745/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:33:26.337454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e0f5e-72de-4618-a38a-d1c4f1cc3a9c None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:26.338308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e0f5e-72de-4618-a38a-d1c4f1cc3a9c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:26.338308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e0f5e-72de-4618-a38a-d1c4f1cc3a9c None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:26.340507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 743/1488] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:33:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:26.347337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451332f-a400-4c60-a51e-aab172b43161 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:26.347871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451332f-a400-4c60-a51e-aab172b43161 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:26.348217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451332f-a400-4c60-a51e-aab172b43161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:26.423267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3451332f-a400-4c60-a51e-aab172b43161 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:26.424511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3451332f-a400-4c60-a51e-aab172b43161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:26.485987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:26.533534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 746/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:26.822326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e0f5e-72de-4618-a38a-d1c4f1cc3a9c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:33:26.823953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ba6e0f5e-72de-4618-a38a-d1c4f1cc3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:26.862382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:26.938552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e0f5e-72de-4618-a38a-d1c4f1cc3a9c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:33:27.019545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171a266-5087-41ad-ac8c-a4a85a499485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:33:27.020209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171a266-5087-41ad-ac8c-a4a85a499485 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:27.020484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171a266-5087-41ad-ac8c-a4a85a499485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:27.020752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171a266-5087-41ad-ac8c-a4a85a499485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:27.022353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0171a266-5087-41ad-ac8c-a4a85a499485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:27.118994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:27.150865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171a266-5087-41ad-ac8c-a4a85a499485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:33:27.152898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171a266-5087-41ad-ac8c-a4a85a499485 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:33:27.172505 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 744/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:33:29.326814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b0b6-6d80-467b-8c14-e6dca560849a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:29.327726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b0b6-6d80-467b-8c14-e6dca560849a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:29.327726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b0b6-6d80-467b-8c14-e6dca560849a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:29.329643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 747/1491] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 22 05:33:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:33:29.336309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6a9a5-0029-4dfc-b3cd-483370270e50 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:29.336747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6a9a5-0029-4dfc-b3cd-483370270e50 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:29.336852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6a9a5-0029-4dfc-b3cd-483370270e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:29.406179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-91d6a9a5-0029-4dfc-b3cd-483370270e50 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:33:29.407289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-91d6a9a5-0029-4dfc-b3cd-483370270e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:29.515192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 745/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 22 05:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:33:29.800099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b0b6-6d80-467b-8c14-e6dca560849a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:33:29.802149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-69d2b0b6-6d80-467b-8c14-e6dca560849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:29.925924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b0b6-6d80-467b-8c14-e6dca560849a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:33:30.015433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c14777-2fd1-4dba-a42b-dbeed3ee8e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:33:30.016081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c14777-2fd1-4dba-a42b-dbeed3ee8e50 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:33:30.016359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c14777-2fd1-4dba-a42b-dbeed3ee8e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:33:30.016645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c14777-2fd1-4dba-a42b-dbeed3ee8e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:33:30.018251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d4c14777-2fd1-4dba-a42b-dbeed3ee8e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:33:30.027959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:30.133002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c14777-2fd1-4dba-a42b-dbeed3ee8e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:33:30.134444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c14777-2fd1-4dba-a42b-dbeed3ee8e50 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:33:30.152556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 748/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:33:39.513214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=82 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:33:40.150218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=50 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:48.202923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:48.203908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:48.203908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:48.288377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:34:48.289583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:48.393751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 746/1494] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:34:48.402222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78eeb-c058-4f06-890e-135db4e51850 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:48.402746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78eeb-c058-4f06-890e-135db4e51850 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:48.402746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78eeb-c058-4f06-890e-135db4e51850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:48.481205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d1b78eeb-c058-4f06-890e-135db4e51850 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:34:48.482320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d1b78eeb-c058-4f06-890e-135db4e51850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:48.590891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 749/1495] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:34:48.598129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:48.599401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:48.611293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97701f71-2c5c-4f44-abfa-fc5aebeb0b1a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:48.678518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:48.713390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:48.714002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:48.714281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:48.714550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:48.715210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:34:48.715510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:48.716563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:48.716563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:34:48.716563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:34:48.717453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:48.730126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 747/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:34:48.743109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b78eeb-c058-4f06-890e-135db4e51850 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:48.752647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b78eeb-c058-4f06-890e-135db4e51850 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:48.815435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:48.851670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:48.852289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:48.852555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:48.852832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:48.853453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:48.855286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING py.warnings [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:48.855286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: warnings.warn(msg) Aug 22 05:34:48.855286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: Aug 22 05:34:48.856509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:48.926503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 750/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:34:48 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:34:48.931328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:48.932929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:48.968107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951bb60-a1d4-49c5-aecb-8f2619d81ae6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:49.061716 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:49.062228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:49.062498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:49.062766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:49.063376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:49.064427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:49.064427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:34:49.064427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:34:49.065624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:49.113001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 748/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:34:48 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:34:49.117506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:49.119029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:49.180226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6cd6f-4084-4258-9286-e44e9f786769 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:49.280931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:49.292834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:49.293451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:49.293728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:49.293998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:49.294648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:49.295716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING py.warnings [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:49.295716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: warnings.warn(msg) Aug 22 05:34:49.295716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: Aug 22 05:34:49.296602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:49.309276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 751/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:34:49.313869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:49.315329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:49.326943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e0690-f731-4562-a66a-ff01b9ebd0ec admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:49.353355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:49.437380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb974de-f972-40e2-9fdf-8c1cef5319cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:49.437988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb974de-f972-40e2-9fdf-8c1cef5319cf admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:49.438263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb974de-f972-40e2-9fdf-8c1cef5319cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:49.438527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb974de-f972-40e2-9fdf-8c1cef5319cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:49.439164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb974de-f972-40e2-9fdf-8c1cef5319cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:49.440220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-2eb974de-f972-40e2-9fdf-8c1cef5319cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:49.440220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:34:49.440220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:34:49.441104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb974de-f972-40e2-9fdf-8c1cef5319cf admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:49.454217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 749/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:34:49.458748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:49.460327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:49.498445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f939db8-eb0f-4198-a8c3-27601e22a625 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:49.597175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd574cf-81aa-49e7-91b9-51156db47eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:49.597749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd574cf-81aa-49e7-91b9-51156db47eee admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:49.598025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd574cf-81aa-49e7-91b9-51156db47eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:49.598305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd574cf-81aa-49e7-91b9-51156db47eee admin admin] PATH_INFO: `/v3/projects/273ad2f9b9134e1ca3fa946c9923ca0e/users/33e6c408fa274df79ddab1bcf87879d9/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:49.620823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd574cf-81aa-49e7-91b9-51156db47eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=273ad2f9b9134e1ca3fa946c9923ca0e, user_id=33e6c408fa274df79ddab1bcf87879d9, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:49.621860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING py.warnings [None req-8fd574cf-81aa-49e7-91b9-51156db47eee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:49.621860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: warnings.warn(msg) Aug 22 05:34:49.621860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: Aug 22 05:34:49.623822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd574cf-81aa-49e7-91b9-51156db47eee admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:49.649970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 752/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:34:49 2022] PUT /identity/v3/projects/273ad2f9b9134e1ca3fa946c9923ca0e/users/33e6c408fa274df79ddab1bcf87879d9/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:34:49.662330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126458a-fc30-446b-9f68-94f979d2e5de None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:49.662612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126458a-fc30-446b-9f68-94f979d2e5de None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:49.662883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126458a-fc30-446b-9f68-94f979d2e5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:49.723855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-2126458a-fc30-446b-9f68-94f979d2e5de None None] MFA Rules not processed for user `33e6c408fa274df79ddab1bcf87879d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:34:49.725111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-2126458a-fc30-446b-9f68-94f979d2e5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:49.809346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 750/1502] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:34:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:34:49.823859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36eff8c-acf4-4eed-a240-4745c22a5bf0 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:49.824151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36eff8c-acf4-4eed-a240-4745c22a5bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:49.824427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36eff8c-acf4-4eed-a240-4745c22a5bf0 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:49.825835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 753/1503] 10.222.0.18 () {54 vars in 964 bytes} [Mon Aug 22 05:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:34:49.830932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f14f21-1dd2-49fa-aab7-aa3e32c5a3e1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:49.831222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f14f21-1dd2-49fa-aab7-aa3e32c5a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:49.831499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f14f21-1dd2-49fa-aab7-aa3e32c5a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:49.863951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:49.909407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-79f14f21-1dd2-49fa-aab7-aa3e32c5a3e1 None None] MFA Rules not processed for user `3fcf79bef8d542d9adbfce89931032c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:34:49.910626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-79f14f21-1dd2-49fa-aab7-aa3e32c5a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:50.001162 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 751/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Mon Aug 22 05:34:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:34:50.011911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36eff8c-acf4-4eed-a240-4745c22a5bf0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:50.013492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d36eff8c-acf4-4eed-a240-4745c22a5bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:50.067612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:50.252954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36eff8c-acf4-4eed-a240-4745c22a5bf0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:50.329604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:50.330095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:50.330365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:50.330611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:50.332272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:50.441055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:50.442509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:50.446204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:50.464188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 754/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:34:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:34:50.513832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ed3a4-fc96-49ec-bb65-06952cbf5fa9 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:50.514312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ed3a4-fc96-49ec-bb65-06952cbf5fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:50.514407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ed3a4-fc96-49ec-bb65-06952cbf5fa9 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:50.515707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 752/1506] 10.222.0.18 () {52 vars in 830 bytes} [Mon Aug 22 05:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:34:50.519380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:50.520639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:50.621402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3cf84-4231-4e32-a251-6a726c01a4b5 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:50.735257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:50.735926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:50.736198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:50.736445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] PATH_INFO: `/v3/users/33e6c408fa274df79ddab1bcf87879d9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:50.751656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e6c408fa274df79ddab1bcf87879d9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:50.752725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING py.warnings [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:50.752725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: warnings.warn(msg) Aug 22 05:34:50.752725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: Aug 22 05:34:50.753981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:50.767236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:50.828610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e6c408fa274df79ddab1bcf87879d9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:50.829046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e6c408fa274df79ddab1bcf87879d9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:50.843329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e6c408fa274df79ddab1bcf87879d9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:50.843910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e6c408fa274df79ddab1bcf87879d9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:50.854586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e6c408fa274df79ddab1bcf87879d9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:50.883050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 755/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:34:50 2022] DELETE /identity/v3/users/33e6c408fa274df79ddab1bcf87879d9 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:34:50.887874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:50.888162 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:50.888439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:50.957476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:34:50.958702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:51.036569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:51.050140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 753/1508] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:34:51.062172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:51.063500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:51.174503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22351e35-c630-469b-942c-170740b484e1 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:51.237483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:34:51.259709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5ea0a-c01b-421a-a99a-9c9a0ac5bc99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:51.260360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5ea0a-c01b-421a-a99a-9c9a0ac5bc99 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:51.260639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5ea0a-c01b-421a-a99a-9c9a0ac5bc99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:51.260911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5ea0a-c01b-421a-a99a-9c9a0ac5bc99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:51.262642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-34a5ea0a-c01b-421a-a99a-9c9a0ac5bc99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:51.377132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5ea0a-c01b-421a-a99a-9c9a0ac5bc99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:51.378591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5ea0a-c01b-421a-a99a-9c9a0ac5bc99 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:51.399180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 756/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:34:51.744738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:34:51.745983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:34:51.757921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705b948-bebc-4c11-8237-0b31b0ad04ab None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:34:51.862712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:34:51.863359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:34:51.863659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:34:51.863932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] PATH_INFO: `/v3/projects/273ad2f9b9134e1ca3fa946c9923ca0e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:34:51.875163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] RBAC: Authorizing `identity:delete_project(project_id=273ad2f9b9134e1ca3fa946c9923ca0e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:34:51.876321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:34:51.876321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:34:51.876321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:34:51.877526 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:34:51.943706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273ad2f9b9134e1ca3fa946c9923ca0e', 'request_id': 'req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:51.950744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273ad2f9b9134e1ca3fa946c9923ca0e', 'request_id': 'req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:51.965262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273ad2f9b9134e1ca3fa946c9923ca0e', 'request_id': 'req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:51.995767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] The token cache is being invalidate because project 273ad2f9b9134e1ca3fa946c9923ca0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:34:51.996095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:51.996604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c367b1c7-1c1b-4ec1-afe2-c50642e9347d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:34:51.997942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 754/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:34:51 2022] DELETE /identity/v3/projects/273ad2f9b9134e1ca3fa946c9923ca0e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:35:01.396907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=62 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:35:01.993298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:36:14.926396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce160ed0-01eb-4757-845c-22dd821201bf None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:14.926954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce160ed0-01eb-4757-845c-22dd821201bf None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:14.926954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce160ed0-01eb-4757-845c-22dd821201bf None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:14.928402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:14.934089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534133e-d53d-4957-97c6-0a42ac779d3f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:14.934559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534133e-d53d-4957-97c6-0a42ac779d3f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:14.934667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534133e-d53d-4957-97c6-0a42ac779d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:15.025642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4534133e-d53d-4957-97c6-0a42ac779d3f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:15.026859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4534133e-d53d-4957-97c6-0a42ac779d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:15.136282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:15.290711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce160ed0-01eb-4757-845c-22dd821201bf None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:15.294030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ce160ed0-01eb-4757-845c-22dd821201bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:15.429520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce160ed0-01eb-4757-845c-22dd821201bf None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:15.437173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdd2a8-350d-4ec7-b483-1b4bb5cb2ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:15.438041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdd2a8-350d-4ec7-b483-1b4bb5cb2ded service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:15.438409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdd2a8-350d-4ec7-b483-1b4bb5cb2ded service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:15.438754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdd2a8-350d-4ec7-b483-1b4bb5cb2ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:15.452416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdd2a8-350d-4ec7-b483-1b4bb5cb2ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:15.454741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdd2a8-350d-4ec7-b483-1b4bb5cb2ded service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:15.467608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 758/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:16.325140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b3ef-a4ce-4f55-bc34-68d6c7751fbd None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:16.325757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b3ef-a4ce-4f55-bc34-68d6c7751fbd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:16.325757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b3ef-a4ce-4f55-bc34-68d6c7751fbd None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:16.327073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:16.333320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd282319-c433-419c-998f-7412786790b9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:16.334072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd282319-c433-419c-998f-7412786790b9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:16.334221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd282319-c433-419c-998f-7412786790b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:16.360462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-fd282319-c433-419c-998f-7412786790b9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:16.361658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fd282319-c433-419c-998f-7412786790b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:16.370137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:16.523515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051b3ef-a4ce-4f55-bc34-68d6c7751fbd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:16.541883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051b3ef-a4ce-4f55-bc34-68d6c7751fbd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:16.548908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735371b8-9485-4ab3-a9ca-d357495d12fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:16.549521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735371b8-9485-4ab3-a9ca-d357495d12fd service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:16.549777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735371b8-9485-4ab3-a9ca-d357495d12fd service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:16.550045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735371b8-9485-4ab3-a9ca-d357495d12fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:16.559376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735371b8-9485-4ab3-a9ca-d357495d12fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:16.561263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735371b8-9485-4ab3-a9ca-d357495d12fd service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:16.576819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 757/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:17.167266 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be11a7-f73d-424a-959c-8c27fadfe238 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:17.167498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be11a7-f73d-424a-959c-8c27fadfe238 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:17.167743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be11a7-f73d-424a-959c-8c27fadfe238 None None] PATH_INFO: `/v3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:17.169079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 760/1517] 10.222.0.15 () {54 vars in 925 bytes} [Mon Aug 22 05:36:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:36:17.176114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ebf5e-c028-445b-955a-4360d69c6659 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:17.176407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ebf5e-c028-445b-955a-4360d69c6659 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:17.176669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ebf5e-c028-445b-955a-4360d69c6659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:17.246046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-2d7ebf5e-c028-445b-955a-4360d69c6659 None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:17.247859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-2d7ebf5e-c028-445b-955a-4360d69c6659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:17.270383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:36:17.334632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 758/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 22 05:36:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:21.213545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada346e4-954d-4638-8c8b-d257f611f843 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:21.214158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada346e4-954d-4638-8c8b-d257f611f843 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:21.214158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada346e4-954d-4638-8c8b-d257f611f843 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:21.215398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 761/1519] 10.222.0.18 () {54 vars in 961 bytes} [Mon Aug 22 05:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:21.221810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a26df9-85fa-4ae2-9e44-b249001780cf None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:21.222262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a26df9-85fa-4ae2-9e44-b249001780cf None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:21.222402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a26df9-85fa-4ae2-9e44-b249001780cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:21.250699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-86a26df9-85fa-4ae2-9e44-b249001780cf None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:21.251959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-86a26df9-85fa-4ae2-9e44-b249001780cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:21.262213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 759/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Mon Aug 22 05:36:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:21.270498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada346e4-954d-4638-8c8b-d257f611f843 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:21.285972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada346e4-954d-4638-8c8b-d257f611f843 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:21.291229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a3e38-cd4a-472e-a894-c054c053e12b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:21.291864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a3e38-cd4a-472e-a894-c054c053e12b service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:21.292154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a3e38-cd4a-472e-a894-c054c053e12b service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:21.292428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a3e38-cd4a-472e-a894-c054c053e12b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:21.300802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a3e38-cd4a-472e-a894-c054c053e12b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:21.302411 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a3e38-cd4a-472e-a894-c054c053e12b service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:21.312292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 762/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:27.824427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22af898-2b0f-4abe-bedd-c9a2116a3b93 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:27.825065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22af898-2b0f-4abe-bedd-c9a2116a3b93 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:27.825065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22af898-2b0f-4abe-bedd-c9a2116a3b93 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:27.826989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:36:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:27.834951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7d3b-6fde-4fa4-9d43-0c8d2437c399 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:27.835783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7d3b-6fde-4fa4-9d43-0c8d2437c399 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:27.835959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7d3b-6fde-4fa4-9d43-0c8d2437c399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:27.858773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9bda7d3b-6fde-4fa4-9d43-0c8d2437c399 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:27.860039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9bda7d3b-6fde-4fa4-9d43-0c8d2437c399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:27.868665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:28.023136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22af898-2b0f-4abe-bedd-c9a2116a3b93 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:28.045894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22af898-2b0f-4abe-bedd-c9a2116a3b93 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:28.051429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284dd66-7bd3-49a5-a3f5-7df62c9c0736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:28.052060 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284dd66-7bd3-49a5-a3f5-7df62c9c0736 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:28.052334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284dd66-7bd3-49a5-a3f5-7df62c9c0736 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:28.052600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284dd66-7bd3-49a5-a3f5-7df62c9c0736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:28.060816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284dd66-7bd3-49a5-a3f5-7df62c9c0736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:28.062408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284dd66-7bd3-49a5-a3f5-7df62c9c0736 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:28.071753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 761/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:28.888843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17b7c-84b9-4e08-ad4c-6b692e7ad1b6 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:28.889526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17b7c-84b9-4e08-ad4c-6b692e7ad1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:28.889526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17b7c-84b9-4e08-ad4c-6b692e7ad1b6 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:28.892519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 05:36:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:28.902575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607c00f-2ce7-4cb1-b628-e5e5cfc8dba2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:28.903137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607c00f-2ce7-4cb1-b628-e5e5cfc8dba2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:28.903756 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607c00f-2ce7-4cb1-b628-e5e5cfc8dba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:28.929601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4607c00f-2ce7-4cb1-b628-e5e5cfc8dba2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:28.930794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4607c00f-2ce7-4cb1-b628-e5e5cfc8dba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:28.938927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 05:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:29.092113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f17b7c-84b9-4e08-ad4c-6b692e7ad1b6 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:29.111033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f17b7c-84b9-4e08-ad4c-6b692e7ad1b6 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:29.117962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8574900-8efd-4595-aad6-a17ea197a081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:29.118560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574900-8efd-4595-aad6-a17ea197a081 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:29.118834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574900-8efd-4595-aad6-a17ea197a081 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:29.119102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574900-8efd-4595-aad6-a17ea197a081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:29.128516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8574900-8efd-4595-aad6-a17ea197a081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:29.130163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8574900-8efd-4595-aad6-a17ea197a081 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:29.140062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 765/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:32.959131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae4a-b192-459d-865b-3198190ba551 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:32.959728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae4a-b192-459d-865b-3198190ba551 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:32.959838 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bae4a-b192-459d-865b-3198190ba551 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:32.961972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:36:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:32.968777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1eb6-b9a4-46cc-83ed-790cfb098a91 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:32.969609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1eb6-b9a4-46cc-83ed-790cfb098a91 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:32.969753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1eb6-b9a4-46cc-83ed-790cfb098a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:32.990958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-560f1eb6-b9a4-46cc-83ed-790cfb098a91 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:32.992207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-560f1eb6-b9a4-46cc-83ed-790cfb098a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:33.001224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:33.296762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bae4a-b192-459d-865b-3198190ba551 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:33.298795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-154bae4a-b192-459d-865b-3198190ba551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:33.400201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bae4a-b192-459d-865b-3198190ba551 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:33.404693 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116eec-9c75-4f65-8c22-1ba5af28ca5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:33.405311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116eec-9c75-4f65-8c22-1ba5af28ca5a service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:33.405593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116eec-9c75-4f65-8c22-1ba5af28ca5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:33.405863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116eec-9c75-4f65-8c22-1ba5af28ca5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:33.415987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a116eec-9c75-4f65-8c22-1ba5af28ca5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:33.417438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a116eec-9c75-4f65-8c22-1ba5af28ca5a service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:33.426937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 764/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:35.302158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1643b69-0a51-4ff4-b5aa-9468059e7587 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:35.302662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1643b69-0a51-4ff4-b5aa-9468059e7587 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:35.302766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1643b69-0a51-4ff4-b5aa-9468059e7587 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:35.304883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:36:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:35.312915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bcbb6-eb8f-441e-8951-52aae2d8995e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:35.313413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bcbb6-eb8f-441e-8951-52aae2d8995e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:35.313530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bcbb6-eb8f-441e-8951-52aae2d8995e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:35.341291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4f4bcbb6-eb8f-441e-8951-52aae2d8995e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:35.342512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4f4bcbb6-eb8f-441e-8951-52aae2d8995e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:35.352279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:35.638316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1643b69-0a51-4ff4-b5aa-9468059e7587 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:35.651674 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1643b69-0a51-4ff4-b5aa-9468059e7587 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:35.656377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097addbf-651f-425b-b76d-207812d2f420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:35.656978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097addbf-651f-425b-b76d-207812d2f420 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:35.657252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097addbf-651f-425b-b76d-207812d2f420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:35.657519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097addbf-651f-425b-b76d-207812d2f420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:35.668108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097addbf-651f-425b-b76d-207812d2f420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:35.669587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097addbf-651f-425b-b76d-207812d2f420 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:35.679334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 768/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:39.220013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17af13-9b0d-4666-87d5-fbfbaa1532da None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:39.220573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17af13-9b0d-4666-87d5-fbfbaa1532da None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:39.220573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17af13-9b0d-4666-87d5-fbfbaa1532da None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:39.221822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:39.228372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3f925-745e-4f75-87cc-cac91deb2bfa None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:39.229087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3f925-745e-4f75-87cc-cac91deb2bfa None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:39.229405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3f925-745e-4f75-87cc-cac91deb2bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:39.251094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2bc3f925-745e-4f75-87cc-cac91deb2bfa None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:39.252329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2bc3f925-745e-4f75-87cc-cac91deb2bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:39.260767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:39.821330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17af13-9b0d-4666-87d5-fbfbaa1532da None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:39.823753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cc17af13-9b0d-4666-87d5-fbfbaa1532da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:39.891180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:36:39.908922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17af13-9b0d-4666-87d5-fbfbaa1532da None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:39.912935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f954a7-1084-4567-96a3-595089d05854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:39.913549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f954a7-1084-4567-96a3-595089d05854 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:39.913828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f954a7-1084-4567-96a3-595089d05854 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:39.914129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f954a7-1084-4567-96a3-595089d05854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:39.923588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f954a7-1084-4567-96a3-595089d05854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:39.925039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f954a7-1084-4567-96a3-595089d05854 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:39.934416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 767/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:41.874646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77b739-0178-445b-a425-a695709e1a72 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:41.875530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77b739-0178-445b-a425-a695709e1a72 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:41.875530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77b739-0178-445b-a425-a695709e1a72 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:41.877643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 05:36:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:36:41.886311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbeb9-0198-4415-b2e8-fe3b89454f84 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:41.886761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbeb9-0198-4415-b2e8-fe3b89454f84 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:41.886872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbeb9-0198-4415-b2e8-fe3b89454f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:41.911827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5e4fbeb9-0198-4415-b2e8-fe3b89454f84 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:36:41.913052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5e4fbeb9-0198-4415-b2e8-fe3b89454f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:41.922376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 05:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:36:42.468591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77b739-0178-445b-a425-a695709e1a72 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:36:42.470903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1b77b739-0178-445b-a425-a695709e1a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:36:42.479064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=84 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:36:42.488330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77b739-0178-445b-a425-a695709e1a72 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:36:42.494603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e16b69-5f69-4bca-aeb5-4ad771de986f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:36:42.495526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e16b69-5f69-4bca-aeb5-4ad771de986f service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:36:42.496001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e16b69-5f69-4bca-aeb5-4ad771de986f service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:36:42.496416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e16b69-5f69-4bca-aeb5-4ad771de986f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:36:42.506404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e16b69-5f69-4bca-aeb5-4ad771de986f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:36:42.507897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e16b69-5f69-4bca-aeb5-4ad771de986f service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:36:42.518079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 771/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:36:51.906524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=11 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:36:52.513827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:05.936970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34971eba-b664-4fd8-9e49-a3f8b3f1f81a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:05.937630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34971eba-b664-4fd8-9e49-a3f8b3f1f81a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:05.937630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34971eba-b664-4fd8-9e49-a3f8b3f1f81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:05.963323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-34971eba-b664-4fd8-9e49-a3f8b3f1f81a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:05.964639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-34971eba-b664-4fd8-9e49-a3f8b3f1f81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:05.980232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 769/1540] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:05.990497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527815a-6930-4886-ab4e-48aa7716c6fc None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:05.991181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527815a-6930-4886-ab4e-48aa7716c6fc None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:05.991181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527815a-6930-4886-ab4e-48aa7716c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.018278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7527815a-6930-4886-ab4e-48aa7716c6fc None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.019815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7527815a-6930-4886-ab4e-48aa7716c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.032190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 772/1541] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.037627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34971eba-b664-4fd8-9e49-a3f8b3f1f81a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.051836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34971eba-b664-4fd8-9e49-a3f8b3f1f81a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.056625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dff5d-52d0-4a98-9788-9aa82691b2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.057280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dff5d-52d0-4a98-9788-9aa82691b2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.057559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dff5d-52d0-4a98-9788-9aa82691b2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.057843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dff5d-52d0-4a98-9788-9aa82691b2c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.058521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dff5d-52d0-4a98-9788-9aa82691b2c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.058831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dff5d-52d0-4a98-9788-9aa82691b2c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.060780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dff5d-52d0-4a98-9788-9aa82691b2c0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.080112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 770/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.089057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527815a-6930-4886-ab4e-48aa7716c6fc None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.099879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527815a-6930-4886-ab4e-48aa7716c6fc None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.104645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.105273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.105556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.105827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.106596 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.108609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.165191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8023d6-53eb-4f76-abac-8017c299c9da None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.165419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8023d6-53eb-4f76-abac-8017c299c9da None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.165644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8023d6-53eb-4f76-abac-8017c299c9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.175155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 773/1543] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/projects => generated 393 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:06.180810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.204325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-2e8023d6-53eb-4f76-abac-8017c299c9da None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.206623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-2e8023d6-53eb-4f76-abac-8017c299c9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.313428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d528-ae38-49e9-810f-11f5c8523047 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.318148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37137e55-5e7f-4ec9-8899-2a2c7c1d56fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.318775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137e55-5e7f-4ec9-8899-2a2c7c1d56fa admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.319056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137e55-5e7f-4ec9-8899-2a2c7c1d56fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.319336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137e55-5e7f-4ec9-8899-2a2c7c1d56fa admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.320845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37137e55-5e7f-4ec9-8899-2a2c7c1d56fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.322060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING py.warnings [None req-37137e55-5e7f-4ec9-8899-2a2c7c1d56fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:38:06.322060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: warnings.warn(msg) Aug 22 05:38:06.322060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: Aug 22 05:38:06.323289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37137e55-5e7f-4ec9-8899-2a2c7c1d56fa admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.327330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 771/1544] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:06.328900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e72c9-b322-4184-b53d-ae71b0231391 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.329412 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e72c9-b322-4184-b53d-ae71b0231391 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.330107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e72c9-b322-4184-b53d-ae71b0231391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.353279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3b7e72c9-b322-4184-b53d-ae71b0231391 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.354547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3b7e72c9-b322-4184-b53d-ae71b0231391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.361609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 774/1545] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:06.364779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617654-1786-4d25-b477-5bfe64d4688e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.364986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 772/1546] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.365770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617654-1786-4d25-b477-5bfe64d4688e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.366305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be8e90-3bbe-4b18-97ce-62facbdd34cb None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.366599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be8e90-3bbe-4b18-97ce-62facbdd34cb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.366858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be8e90-3bbe-4b18-97ce-62facbdd34cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.367225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617654-1786-4d25-b477-5bfe64d4688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.391068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-23be8e90-3bbe-4b18-97ce-62facbdd34cb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.391856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-59617654-1786-4d25-b477-5bfe64d4688e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.392320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-23be8e90-3bbe-4b18-97ce-62facbdd34cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.393016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-59617654-1786-4d25-b477-5bfe64d4688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.400179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 773/1547] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.401499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc731cde-f69f-4ee1-8e8e-3f231bfc0555 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.401787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc731cde-f69f-4ee1-8e8e-3f231bfc0555 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.402058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc731cde-f69f-4ee1-8e8e-3f231bfc0555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.408006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 775/1548] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.413002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8233f-7573-4c37-97d3-294a2dfd2720 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.413265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8233f-7573-4c37-97d3-294a2dfd2720 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.413555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8233f-7573-4c37-97d3-294a2dfd2720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.423984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-cc731cde-f69f-4ee1-8e8e-3f231bfc0555 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.425155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cc731cde-f69f-4ee1-8e8e-3f231bfc0555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.432585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-dbe8233f-7573-4c37-97d3-294a2dfd2720 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.433175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 774/1549] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.433771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-dbe8233f-7573-4c37-97d3-294a2dfd2720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.434569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7310b-85c0-4db7-b58d-06ace10a89f2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.434840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7310b-85c0-4db7-b58d-06ace10a89f2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.435116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7310b-85c0-4db7-b58d-06ace10a89f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.442640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 776/1550] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.443460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8233f-7573-4c37-97d3-294a2dfd2720 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.454666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-23b7310b-85c0-4db7-b58d-06ace10a89f2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.455898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-23b7310b-85c0-4db7-b58d-06ace10a89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.457693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8233f-7573-4c37-97d3-294a2dfd2720 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.462425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b4753-dc75-466b-99dc-84b8b7005c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.463033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4753-dc75-466b-99dc-84b8b7005c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.463311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4753-dc75-466b-99dc-84b8b7005c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.463595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4753-dc75-466b-99dc-84b8b7005c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.463755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 775/1551] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.464241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b4753-dc75-466b-99dc-84b8b7005c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.464915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b1f9-c47a-44da-b3e0-08f47c8f013c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.465192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b1f9-c47a-44da-b3e0-08f47c8f013c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.465517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b1f9-c47a-44da-b3e0-08f47c8f013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.466066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b4753-dc75-466b-99dc-84b8b7005c96 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.479773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 777/1552] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:06.479773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425a903-032b-4116-9f0b-d5a70daad2a1 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.479773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425a903-032b-4116-9f0b-d5a70daad2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.479773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425a903-032b-4116-9f0b-d5a70daad2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.485775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ef42b1f9-c47a-44da-b3e0-08f47c8f013c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.487594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ef42b1f9-c47a-44da-b3e0-08f47c8f013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.495299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 776/1553] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.496514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d12536-c17c-409e-b1bf-1fbe25bf69aa None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.496873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d12536-c17c-409e-b1bf-1fbe25bf69aa None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.497139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d12536-c17c-409e-b1bf-1fbe25bf69aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.498913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5425a903-032b-4116-9f0b-d5a70daad2a1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.500144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5425a903-032b-4116-9f0b-d5a70daad2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.508725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 778/1554] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.509994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd659478-9855-459a-8dd1-8fb43e85e30e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.510262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd659478-9855-459a-8dd1-8fb43e85e30e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.510554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd659478-9855-459a-8dd1-8fb43e85e30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.518157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-94d12536-c17c-409e-b1bf-1fbe25bf69aa None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.520126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-94d12536-c17c-409e-b1bf-1fbe25bf69aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.527736 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 777/1555] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.529255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072f0d1-50a8-462d-8b52-dc03b2325646 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.529540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072f0d1-50a8-462d-8b52-dc03b2325646 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.529811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072f0d1-50a8-462d-8b52-dc03b2325646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.530125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-dd659478-9855-459a-8dd1-8fb43e85e30e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.531562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-dd659478-9855-459a-8dd1-8fb43e85e30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.539417 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 779/1556] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.540698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc70a75-f840-4270-a928-8a1286c7f82e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.541005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc70a75-f840-4270-a928-8a1286c7f82e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.541278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc70a75-f840-4270-a928-8a1286c7f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.549004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9072f0d1-50a8-462d-8b52-dc03b2325646 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.550207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9072f0d1-50a8-462d-8b52-dc03b2325646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.558126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 778/1557] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.558887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072f0d1-50a8-462d-8b52-dc03b2325646 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.560958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5fc70a75-f840-4270-a928-8a1286c7f82e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.562294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5fc70a75-f840-4270-a928-8a1286c7f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.568179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072f0d1-50a8-462d-8b52-dc03b2325646 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.570590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 780/1558] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.571433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc70a75-f840-4270-a928-8a1286c7f82e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.572860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e233fc9-806c-42bd-b5f5-37aa30312ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.573451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e233fc9-806c-42bd-b5f5-37aa30312ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.573725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e233fc9-806c-42bd-b5f5-37aa30312ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.574082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e233fc9-806c-42bd-b5f5-37aa30312ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.574725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e233fc9-806c-42bd-b5f5-37aa30312ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.575063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e233fc9-806c-42bd-b5f5-37aa30312ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.576680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e233fc9-806c-42bd-b5f5-37aa30312ff7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.581100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc70a75-f840-4270-a928-8a1286c7f82e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.586238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.586833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.587110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.587384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.588075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.595829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 779/1559] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.595829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94bb9a-8de1-482f-82aa-7e22d18bbbd6 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.595829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94bb9a-8de1-482f-82aa-7e22d18bbbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.596186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.596297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94bb9a-8de1-482f-82aa-7e22d18bbbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.605861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 781/1560] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:06.606902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.609072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ab94bb9a-8de1-482f-82aa-7e22d18bbbd6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:06.610255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ab94bb9a-8de1-482f-82aa-7e22d18bbbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:06.616924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdccd7-f732-49ad-8fa7-b0a3987671da admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.618010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 780/1561] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:06.618788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94bb9a-8de1-482f-82aa-7e22d18bbbd6 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.625269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.625895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.626174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.626450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.627101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.627407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.627861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94bb9a-8de1-482f-82aa-7e22d18bbbd6 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.628462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING py.warnings [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:38:06.628462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: warnings.warn(msg) Aug 22 05:38:06.628462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: Aug 22 05:38:06.629515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.632276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.632857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.633131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.633401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.634035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.634330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.635936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.641714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 782/1562] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.642792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.648221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 781/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.649234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.654232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4194c1-b91c-4b10-b988-42da90bfebfe admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.659149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.659748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.660056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.660321 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.660931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.661261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.662835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.665530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8875f9-7183-47bc-b517-44677b8e9e8b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.670306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.670919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.671203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.671485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.672174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.672484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.674211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.674721 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 783/1564] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.681923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.685878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86604a-1693-4278-97ce-791df528f2ec admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.687799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 782/1565] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.688682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.697267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.697924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.698195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.698391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021131b-b209-48f9-95f6-4528dc210fff admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.698497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.699134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.699467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.701519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.703308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.703955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.704267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.704541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.705176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.706474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:38:06.706474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:38:06.706474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:38:06.707855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.714288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 784/1566] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.715489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.725880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228152f-0552-4728-b3d0-e20d49f9bfb9 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.731554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.732349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.732696 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.732954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] PATH_INFO: `/v3/projects/8cadb48afce14e758bfe62771530527a/users/9643b0f10e8345889c181efbb1978237/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.747127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 783/1567] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:06.747127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.748585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cadb48afce14e758bfe62771530527a, user_id=9643b0f10e8345889c181efbb1978237, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.751319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.753641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:06.766035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 785/1568] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:06 2022] PUT /identity/v3/projects/8cadb48afce14e758bfe62771530527a/users/9643b0f10e8345889c181efbb1978237/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:06.767004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.775387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:06.814193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea02ec1-7db3-4197-b965-ec23d79e2e11 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.828193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba99cdd-a0a3-4397-a5d3-a5cee545e1e8 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.833004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.833641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.833917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.834190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.834933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.837064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.866415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 786/1569] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:06.867792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.870335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.870959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.871237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.871507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.872511 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:06.872809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.874590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.887050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 784/1570] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:06.888556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:06.901407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900acf62-3297-4d3a-aa59-332474e64d16 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.933305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9030b19-4350-49bf-b4aa-f1fb914cf169 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:06.938199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.938939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.939288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.939651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.940272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.942268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.962834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:06.963485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:06.963795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:06.964065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:06.964704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:06.966639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:06.970129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 787/1571] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:06.970931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.000735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 785/1572] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.000735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.032374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b256b-0668-410b-ac18-05f78ee3caba admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.040113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:07.049775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:07.060593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85067863-54a3-43b7-af49-29a9bccd0e16 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.065569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.066262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.066541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.066813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.067475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.069420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.088393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307d25d-7acb-47f7-8a23-d11d11aac195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.089077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307d25d-7acb-47f7-8a23-d11d11aac195 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.089365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307d25d-7acb-47f7-8a23-d11d11aac195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.089648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307d25d-7acb-47f7-8a23-d11d11aac195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.090260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307d25d-7acb-47f7-8a23-d11d11aac195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.092350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307d25d-7acb-47f7-8a23-d11d11aac195 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.099258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 786/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.100099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.121262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 788/1574] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:06 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.122771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8ada-6d81-4499-8985-60beae3ebff8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.123085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8ada-6d81-4499-8985-60beae3ebff8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.123346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8ada-6d81-4499-8985-60beae3ebff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.146011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-120f8ada-6d81-4499-8985-60beae3ebff8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:07.147236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-120f8ada-6d81-4499-8985-60beae3ebff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:07.169956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbfda7-e6b1-4bd3-a138-1a39e80476ca admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.206509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 789/1575] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:07.210559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f8ada-6d81-4499-8985-60beae3ebff8 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.221499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f8ada-6d81-4499-8985-60beae3ebff8 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.226726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.227354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.227663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.227933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.228562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.228979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.229637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.229919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.230191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.230601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.230823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.232895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.254939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 790/1576] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/users => generated 291 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.255706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 787/1577] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.255987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.256460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.266024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba5b70-753c-435f-8b24-8e2b5783bff6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.266521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfa86f-2c19-4f19-86cf-a0fd7ec5236f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.272465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.272778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.273190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.273392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.273502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.273642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.273737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.273914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.274344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.274538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.276319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.276474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.299399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 788/1578] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/users => generated 291 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.300321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.307094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 791/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.310217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e402720-dfc0-4577-af2f-d901632ba9c8 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.311403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.363805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:07.369175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:07.370510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.371435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.372067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.372579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.372952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.373319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e846ada-87b6-40c6-b422-a02893598177 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.375117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.378855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.379470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.379777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.380057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.380684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.382624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.398962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 789/1580] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.399819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.403746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 792/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.404581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.409746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2755c3-cfdd-48c1-9ddf-8d94d224c4c2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.414183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.414779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.415054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.415340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.415981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.417900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.441409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 790/1582] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/users => generated 284 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.442253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.451884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71bbe2-0e87-4553-91bf-967848529ff6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.456831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.457359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.457641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.457912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.458561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.460266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.463305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e60a9-a534-41e6-8d2c-acd87a875a72 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.467728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.468342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.468972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.469238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.471388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 791/1583] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.474448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.478502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.479969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.482449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829e3a-7487-4258-bf6f-df59b8ef2f5a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.487460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.488606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.488907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.489234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.489835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 793/1584] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:07.489935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.490657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.491505 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.500610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c722a4d-aff1-4283-9454-101aba9fe391 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.503965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 792/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.507005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.507439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.507808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.508018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.508156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.508773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.510511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.520283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1825d-aaa9-4767-9628-af046418d330 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.521946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 794/1586] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.521946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.529492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.530232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.530849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.531214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.531994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.532651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279597d-daee-4dd6-a55d-d98543ec49b1 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.534088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.537547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.538227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.538574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.538942 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.539689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.541335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.555924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 795/1587] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.555924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.564275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1fa99-cb47-40f0-b368-cb1a5dcdcdd7 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.568275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 793/1588] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:07 2022] POST /identity/v3/users => generated 286 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:07.568275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.573073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.573824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.574182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.574513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.575356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.577152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.581692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033f935-416b-464d-87f0-6c586d659cea admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.591825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 796/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.592031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.592150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.592265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.592265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.592265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.592265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.592265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.602907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c342de3-0296-43d2-b4ef-2ef03f0d22ef admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.608010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.608742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.609836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 794/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.609973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.609973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.610256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.610491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.612227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.621764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbee9e2-2f28-4202-8b47-6347d2369d83 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.623253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 797/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.624190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.630687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.631481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.631878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.632232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.632923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.634635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.634948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b2c73-1e24-4b3f-936b-dd7a96fd418c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.641102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.641818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.642165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.642503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.643230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.645200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.647759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 795/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.648874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.657911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 798/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.658558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.662914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aa296-678e-435b-ab5e-9b4afe9fa409 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.667545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.668358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.668715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.669061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.669865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.671479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb1d88-75b9-4fc0-8ad6-22c2de77916e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.671724 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.677476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.678152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.678430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.678699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.679397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.681327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.688318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 796/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.689291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.694247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 799/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.695408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.701332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9ec04-2bb1-4a6d-ac4b-b440664af721 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.709096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.709208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d48e5-f88a-4d50-9cd3-614c256aa68d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.709298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.709298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.711824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.712652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.714173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.714429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.715288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.716217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.716460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.717241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.719745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.733481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 797/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.734335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 800/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:07.734889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.735330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.745877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbce48-4cdb-48ad-8e3e-581fd5081f05 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.750184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6d499-6a19-4694-bacc-4ba6babb7d8a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.752040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.752794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.753142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.753485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] PATH_INFO: `/v3/projects/822f0c2e66354549aeb13d73c512b09d/users/904630cc2e60419d88afd9cfc1c5914d/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.754927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.755629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.755916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.756196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] PATH_INFO: `/v3/projects/c9f504dcd9e447c3b147c90f424544f7/users/6c1bf81b1ab445408ea39d621874e274/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.772898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822f0c2e66354549aeb13d73c512b09d, user_id=904630cc2e60419d88afd9cfc1c5914d, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.773767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9f504dcd9e447c3b147c90f424544f7, user_id=6c1bf81b1ab445408ea39d621874e274, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.774914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:38:07.774914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:38:07.774914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:38:07.776033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.777179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.792981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 801/1598] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:07 2022] PUT /identity/v3/projects/822f0c2e66354549aeb13d73c512b09d/users/904630cc2e60419d88afd9cfc1c5914d/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:07.794204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.803606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 798/1599] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:07 2022] PUT /identity/v3/projects/c9f504dcd9e447c3b147c90f424544f7/users/6c1bf81b1ab445408ea39d621874e274/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:07.803606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.869837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:07.882037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da27a-355b-47cb-afef-c2c06588ac0a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.883243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c08ec-d698-4648-987f-ae800a186a45 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:07.888475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.889201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.889502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.889776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] PATH_INFO: `/v3/projects/b6bddea5c8484dab9602d2eb375aafc1/users/c7bc901f44404ae48fb265243d8ffdcb/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.891504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:07.892393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:07.892817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:07.893243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] PATH_INFO: `/v3/projects/c37fd468d46f4348b13b002e26f2ee3a/users/7395ad97ab494323a7f92ea53b3d910a/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:07.909121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bddea5c8484dab9602d2eb375aafc1, user_id=c7bc901f44404ae48fb265243d8ffdcb, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.911904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.914274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37fd468d46f4348b13b002e26f2ee3a, user_id=7395ad97ab494323a7f92ea53b3d910a, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:07.917516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:07.931052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 799/1600] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:07 2022] PUT /identity/v3/projects/b6bddea5c8484dab9602d2eb375aafc1/users/c7bc901f44404ae48fb265243d8ffdcb/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:07.944893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.947655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 802/1601] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:07 2022] PUT /identity/v3/projects/c37fd468d46f4348b13b002e26f2ee3a/users/7395ad97ab494323a7f92ea53b3d910a/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:07.948861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:07.961447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:08.025959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dafd3-a822-4503-b9bc-5a8d417171e0 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:08.031790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb1cb5-4f35-4d6c-85e4-80da6676e046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:08.032532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb1cb5-4f35-4d6c-85e4-80da6676e046 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.033071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb1cb5-4f35-4d6c-85e4-80da6676e046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.033163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb1cb5-4f35-4d6c-85e4-80da6676e046 admin admin] PATH_INFO: `/v3/projects/d6d9cabf2ed442e7b589b5aff223ef1b/users/416ec4526eb244de813a87210c850da4/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.034200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86bc7b-36c1-491c-a556-700783eae0d3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:08.040905 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:08.041782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.041987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.042237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.042905 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:08.045515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:08.051337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb1cb5-4f35-4d6c-85e4-80da6676e046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d9cabf2ed442e7b589b5aff223ef1b, user_id=416ec4526eb244de813a87210c850da4, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:08.054131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb1cb5-4f35-4d6c-85e4-80da6676e046 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:08.059623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 800/1602] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:08.060729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:08.082872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945cdb2-915c-4f44-8cb4-9a4ec1f53ce9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:08.083086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 803/1603] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:07 2022] PUT /identity/v3/projects/d6d9cabf2ed442e7b589b5aff223ef1b/users/416ec4526eb244de813a87210c850da4/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:08.083086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edbf4f-95d7-4580-bf5f-a1ca3c2a4f9f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.083086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edbf4f-95d7-4580-bf5f-a1ca3c2a4f9f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.083086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edbf4f-95d7-4580-bf5f-a1ca3c2a4f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.112042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-67edbf4f-95d7-4580-bf5f-a1ca3c2a4f9f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:08.113287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-67edbf4f-95d7-4580-bf5f-a1ca3c2a4f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:08.159335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ec26f-dfda-4b19-957a-c11cb3f37268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:08.160310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec26f-dfda-4b19-957a-c11cb3f37268 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.160774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec26f-dfda-4b19-957a-c11cb3f37268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.161047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec26f-dfda-4b19-957a-c11cb3f37268 admin admin] PATH_INFO: `/v3/projects/8e8481f1683045d4a63b3d33e0dab620/users/8e68f520aaff495abfb4b7a5271936e1/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.177630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ec26f-dfda-4b19-957a-c11cb3f37268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8481f1683045d4a63b3d33e0dab620, user_id=8e68f520aaff495abfb4b7a5271936e1, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:08.180641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ec26f-dfda-4b19-957a-c11cb3f37268 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:08.192081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 804/1604] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:08.195315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f96f-4e78-4618-993c-4f5b794a3956 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.195315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f96f-4e78-4618-993c-4f5b794a3956 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.195315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f96f-4e78-4618-993c-4f5b794a3956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.216479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 801/1605] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:08 2022] PUT /identity/v3/projects/8e8481f1683045d4a63b3d33e0dab620/users/8e68f520aaff495abfb4b7a5271936e1/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:08.218337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9677d-724d-464e-b4c0-6d56ad9dce28 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.219051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9677d-724d-464e-b4c0-6d56ad9dce28 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.219320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9677d-724d-464e-b4c0-6d56ad9dce28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.232841 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-89a6f96f-4e78-4618-993c-4f5b794a3956 None None] MFA Rules not processed for user `6c1bf81b1ab445408ea39d621874e274`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:08.234609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-89a6f96f-4e78-4618-993c-4f5b794a3956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:08.256324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-dda9677d-724d-464e-b4c0-6d56ad9dce28 None None] MFA Rules not processed for user `c7bc901f44404ae48fb265243d8ffdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:08.258023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-dda9677d-724d-464e-b4c0-6d56ad9dce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:08.310878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:08.348727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 805/1606] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:08.353731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.354017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.354293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.365184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 802/1607] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:08.366061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9677d-724d-464e-b4c0-6d56ad9dce28 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:08.415842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:08.458438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] MFA Rules not processed for user `7395ad97ab494323a7f92ea53b3d910a`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:08.460406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:08.509776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9677d-724d-464e-b4c0-6d56ad9dce28 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:08.516330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800239-b631-41e1-a079-0190ddc9b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:08.517037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800239-b631-41e1-a079-0190ddc9b24b admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.517317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800239-b631-41e1-a079-0190ddc9b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.517591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800239-b631-41e1-a079-0190ddc9b24b admin admin] PATH_INFO: `/v3/projects/68eebfe8c9654799a66fb3f9f09578b1/users/640b03dc796542249c0c6fa7c86e7fe9/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.563941 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7800239-b631-41e1-a079-0190ddc9b24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68eebfe8c9654799a66fb3f9f09578b1, user_id=640b03dc796542249c0c6fa7c86e7fe9, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:08.568191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7800239-b631-41e1-a079-0190ddc9b24b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:08.596676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 803/1608] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:08 2022] PUT /identity/v3/projects/68eebfe8c9654799a66fb3f9f09578b1/users/640b03dc796542249c0c6fa7c86e7fe9/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:08.596676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cec03f-30c4-4a23-b2c9-ad17a7d6ea01 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.596676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cec03f-30c4-4a23-b2c9-ad17a7d6ea01 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.596676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cec03f-30c4-4a23-b2c9-ad17a7d6ea01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.615088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 806/1609] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:08.615088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:08.615088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:08.661800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-71cec03f-30c4-4a23-b2c9-ad17a7d6ea01 None None] MFA Rules not processed for user `416ec4526eb244de813a87210c850da4`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:08.668178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-71cec03f-30c4-4a23-b2c9-ad17a7d6ea01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:08.695102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78e2c0-6911-4097-8a26-cd8499246dde None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:08.756217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 804/1610] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:08.757717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68215e1-b00c-4acb-828d-ddbdd0a82255 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.758073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68215e1-b00c-4acb-828d-ddbdd0a82255 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.758666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68215e1-b00c-4acb-828d-ddbdd0a82255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.784642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a68215e1-b00c-4acb-828d-ddbdd0a82255 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:08.784968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5006f-4c15-4efa-b6e8-40c53b06f355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:08.785654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5006f-4c15-4efa-b6e8-40c53b06f355 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.785971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5006f-4c15-4efa-b6e8-40c53b06f355 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.786299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5006f-4c15-4efa-b6e8-40c53b06f355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.786427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a68215e1-b00c-4acb-828d-ddbdd0a82255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:08.855826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:08.857586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 805/1611] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:08.858650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68215e1-b00c-4acb-828d-ddbdd0a82255 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:08.872010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68215e1-b00c-4acb-828d-ddbdd0a82255 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:08.879147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a69faf-a04f-426d-ab10-a5285ff25412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:08.879811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a69faf-a04f-426d-ab10-a5285ff25412 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.880091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a69faf-a04f-426d-ab10-a5285ff25412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.880732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a69faf-a04f-426d-ab10-a5285ff25412 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.881928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a69faf-a04f-426d-ab10-a5285ff25412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:08.884179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a69faf-a04f-426d-ab10-a5285ff25412 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:08.898907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5006f-4c15-4efa-b6e8-40c53b06f355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:08.900960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5006f-4c15-4efa-b6e8-40c53b06f355 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:08.918772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 807/1612] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:08.931188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695349-29c1-4e74-a672-6990bda6b06a None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.931732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 806/1613] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:08.933169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85f61-74f8-4526-a636-1cf5c01391fd None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.933454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85f61-74f8-4526-a636-1cf5c01391fd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.933731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85f61-74f8-4526-a636-1cf5c01391fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.935482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695349-29c1-4e74-a672-6990bda6b06a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.935832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695349-29c1-4e74-a672-6990bda6b06a None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.937411 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 808/1614] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 22 05:38:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:38:08.943384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50938844-fbcf-409d-af7b-ba8fc128d5b4 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:08.943695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50938844-fbcf-409d-af7b-ba8fc128d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:08.943971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50938844-fbcf-409d-af7b-ba8fc128d5b4 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:08.951020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 809/1615] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 22 05:38:08 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:38:08.951971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50938844-fbcf-409d-af7b-ba8fc128d5b4 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:08.953997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:08.995264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-8ca85f61-74f8-4526-a636-1cf5c01391fd None None] MFA Rules not processed for user `640b03dc796542249c0c6fa7c86e7fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:08.998012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8ca85f61-74f8-4526-a636-1cf5c01391fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:09.067314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50938844-fbcf-409d-af7b-ba8fc128d5b4 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:09.074123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:09.075341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:09.075989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:09.076273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:09.109253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 807/1616] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:09.110479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca85f61-74f8-4526-a636-1cf5c01391fd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:09.199619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca85f61-74f8-4526-a636-1cf5c01391fd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:09.207147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6ad11-c5bc-44b0-afa6-c2b70b046222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:09.207830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6ad11-c5bc-44b0-afa6-c2b70b046222 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:09.208211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6ad11-c5bc-44b0-afa6-c2b70b046222 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:09.208679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6ad11-c5bc-44b0-afa6-c2b70b046222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:09.224013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:09.226768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:09.240257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 810/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:09.241380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:09.316002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6ad11-c5bc-44b0-afa6-c2b70b046222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:09.318257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6ad11-c5bc-44b0-afa6-c2b70b046222 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:09.360259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 808/1618] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:09.361708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798c90-3415-4150-825d-f685120141a3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:09.364137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798c90-3415-4150-825d-f685120141a3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:09.364479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798c90-3415-4150-825d-f685120141a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:09.369576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29867138-31ad-4b90-aae1-b2507524b093 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:09.375340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608e8f1-c448-4ef8-925e-0555a6cded28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:09.377483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e8f1-c448-4ef8-925e-0555a6cded28 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:09.378117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e8f1-c448-4ef8-925e-0555a6cded28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:09.379473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e8f1-c448-4ef8-925e-0555a6cded28 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:09.381661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608e8f1-c448-4ef8-925e-0555a6cded28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:09.385865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608e8f1-c448-4ef8-925e-0555a6cded28 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:09.436456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 811/1619] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:09 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:09.438062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4711bb4-7c94-4ff1-8cf0-66013f993174 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:09.438638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4711bb4-7c94-4ff1-8cf0-66013f993174 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:09.439383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4711bb4-7c94-4ff1-8cf0-66013f993174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:09.471206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ea798c90-3415-4150-825d-f685120141a3 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:09.472467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ea798c90-3415-4150-825d-f685120141a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:09.480938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e4711bb4-7c94-4ff1-8cf0-66013f993174 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:09.484893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e4711bb4-7c94-4ff1-8cf0-66013f993174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:09.528611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:09.538929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:09.629711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 812/1620] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 22 05:38:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:09.635460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4711bb4-7c94-4ff1-8cf0-66013f993174 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:09.657406 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 809/1621] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 22 05:38:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:09.658904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea798c90-3415-4150-825d-f685120141a3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:09.660633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4711bb4-7c94-4ff1-8cf0-66013f993174 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:09.665589 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:09.666375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:09.666643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:09.667021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:09.667968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:09.675080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:09.678516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea798c90-3415-4150-825d-f685120141a3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:09.700832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:09.701242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:09.701242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:09.702025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:09.702168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:09.704062 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:09.734805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 813/1622] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:09 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:09.735041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:09.741561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 810/1623] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:09.745667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:09.788170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eacef-1178-4fa2-8e93-10a256b550c4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:09.955970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe757-aa6f-4c36-aee9-562e6b56b986 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:10.049212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:10.049212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.049212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.049212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.051124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:10.051124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.051124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.051124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.112616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:10.115312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:10.127684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:10.128946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 811/1624] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:10.129353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:10.130662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:10.155767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 814/1625] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:10.156705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:10.216989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:10.251403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:10.321482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3884ac5-1c35-4dc6-abe7-0ceaac965d58 service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:10.328129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda0918-98ee-4049-a7c0-b6998b653b57 service nova] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:10.345088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:10.345802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.346086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.346414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] PATH_INFO: `/v3/projects/ac81afde9297428a8317b1bdf234f7d9/users/00f29ca575df4ba9b489a0a52a9e7820/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.353708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:10.357172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.357478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.357755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.362003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:10.365301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:10.404638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac81afde9297428a8317b1bdf234f7d9, user_id=00f29ca575df4ba9b489a0a52a9e7820, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:10.418344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:10.445828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 812/1626] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:10 2022] POST /identity/v3/users => generated 293 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:10.451050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:10.478162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 815/1627] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:10 2022] PUT /identity/v3/projects/ac81afde9297428a8317b1bdf234f7d9/users/00f29ca575df4ba9b489a0a52a9e7820/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:10.482258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:10.503085 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1ed4-4801-441f-a306-caa90bbefadc admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:10.594322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d8f8c-9e68-4a2d-bba5-fe97cdf6a928 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:10.607479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532586b5-892c-4d8a-b27d-3ed8f2b28220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:10.608818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532586b5-892c-4d8a-b27d-3ed8f2b28220 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.609358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532586b5-892c-4d8a-b27d-3ed8f2b28220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.609891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532586b5-892c-4d8a-b27d-3ed8f2b28220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.610753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532586b5-892c-4d8a-b27d-3ed8f2b28220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:10.612837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532586b5-892c-4d8a-b27d-3ed8f2b28220 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:10.640877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 816/1628] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:10.641816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbae23-eeb2-4579-bc6c-51ef1fbac59b None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.641950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbae23-eeb2-4579-bc6c-51ef1fbac59b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.642162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbae23-eeb2-4579-bc6c-51ef1fbac59b None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.643656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 817/1629] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 22 05:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:38:10.654379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbae23-eeb2-4579-bc6c-51ef1fbac59b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:10.680292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbae23-eeb2-4579-bc6c-51ef1fbac59b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:10.692412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20121e89-5b65-4bfa-bfe8-7f872603242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:10.693236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20121e89-5b65-4bfa-bfe8-7f872603242f admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.693888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20121e89-5b65-4bfa-bfe8-7f872603242f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.693888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20121e89-5b65-4bfa-bfe8-7f872603242f admin admin] PATH_INFO: `/v3/projects/ac81afde9297428a8317b1bdf234f7d9/users/00f29ca575df4ba9b489a0a52a9e7820/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.697325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20121e89-5b65-4bfa-bfe8-7f872603242f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac81afde9297428a8317b1bdf234f7d9, user_id=00f29ca575df4ba9b489a0a52a9e7820, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:10.702395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20121e89-5b65-4bfa-bfe8-7f872603242f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:10.735737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 818/1630] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:10 2022] PUT /identity/v3/projects/ac81afde9297428a8317b1bdf234f7d9/users/00f29ca575df4ba9b489a0a52a9e7820/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:10.735737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb18c-36c7-4ac4-a19f-32cc3baae537 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.735737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb18c-36c7-4ac4-a19f-32cc3baae537 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.735737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb18c-36c7-4ac4-a19f-32cc3baae537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.755046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0800f3-0bee-430f-860b-898e75bbe02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:10.756647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0800f3-0bee-430f-860b-898e75bbe02c admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.758069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0800f3-0bee-430f-860b-898e75bbe02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.758395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0800f3-0bee-430f-860b-898e75bbe02c admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.759156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0800f3-0bee-430f-860b-898e75bbe02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:10.761496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0800f3-0bee-430f-860b-898e75bbe02c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:10.772786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-587bb18c-36c7-4ac4-a19f-32cc3baae537 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:10.774111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-587bb18c-36c7-4ac4-a19f-32cc3baae537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:10.782427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 813/1631] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:10.784084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3af3c-a3af-4d99-b6eb-13e4854e71d5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.784369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3af3c-a3af-4d99-b6eb-13e4854e71d5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.784907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3af3c-a3af-4d99-b6eb-13e4854e71d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:10.839246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-63b3af3c-a3af-4d99-b6eb-13e4854e71d5 None None] MFA Rules not processed for user `00f29ca575df4ba9b489a0a52a9e7820`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:10.841517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-63b3af3c-a3af-4d99-b6eb-13e4854e71d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:10.930500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 819/1632] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:38:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:10.931600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb18c-36c7-4ac4-a19f-32cc3baae537 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:10.980800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 814/1633] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:10.982692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a9c4-185e-48b6-8327-817a3ab0866d None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:10.983004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a9c4-185e-48b6-8327-817a3ab0866d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:10.983261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a9c4-185e-48b6-8327-817a3ab0866d None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.010833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 815/1634] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 22 05:38:10 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:38:11.011884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3a9c4-185e-48b6-8327-817a3ab0866d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:11.012677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:11.082991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb18c-36c7-4ac4-a19f-32cc3baae537 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:11.089730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff375937-c70f-44a8-92d2-7b063ebfeeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:11.091210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff375937-c70f-44a8-92d2-7b063ebfeeb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.091515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff375937-c70f-44a8-92d2-7b063ebfeeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.091879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff375937-c70f-44a8-92d2-7b063ebfeeb8 admin admin] PATH_INFO: `/v3/projects/e2651c5c5cdf49a28fb89030ca05e2fd/users/dca0d7ea822a475aa7747b61d51fa23b/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.114338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff375937-c70f-44a8-92d2-7b063ebfeeb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2651c5c5cdf49a28fb89030ca05e2fd, user_id=dca0d7ea822a475aa7747b61d51fa23b, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:11.117341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff375937-c70f-44a8-92d2-7b063ebfeeb8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:11.150243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 820/1635] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:10 2022] PUT /identity/v3/projects/e2651c5c5cdf49a28fb89030ca05e2fd/users/dca0d7ea822a475aa7747b61d51fa23b/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:11.153587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935412f1-e656-4743-9a9a-dc2ea864f196 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.153882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935412f1-e656-4743-9a9a-dc2ea864f196 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.154197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935412f1-e656-4743-9a9a-dc2ea864f196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.165542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3a9c4-185e-48b6-8327-817a3ab0866d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:11.208704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-935412f1-e656-4743-9a9a-dc2ea864f196 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:11.211556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:11.212100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-935412f1-e656-4743-9a9a-dc2ea864f196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:11.313923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:11.314559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.314832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.315099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.410806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 821/1636] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:38:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:11.412065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935412f1-e656-4743-9a9a-dc2ea864f196 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:11.432776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935412f1-e656-4743-9a9a-dc2ea864f196 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:11.441981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:11.442624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.442899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.443163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.497721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:11.499208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:11.525181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 816/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:11.532228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:11.588432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:11.589898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:11.611790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 822/1638] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:11.665430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20e823-1722-4c67-a83e-19611a1ae1aa service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:11.681310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:11.682705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.684050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.684497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.686809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:11.692324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:11.717577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 817/1639] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:11.722175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:11.745262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:11.750847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9322d1-7e26-484d-a861-63d1fe3ecdcf service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:11.770123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef5390-1ccc-4dd2-9300-3eb2a0945548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:11.770829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef5390-1ccc-4dd2-9300-3eb2a0945548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.771097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef5390-1ccc-4dd2-9300-3eb2a0945548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.771365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef5390-1ccc-4dd2-9300-3eb2a0945548 admin admin] PATH_INFO: `/v3/projects/e2651c5c5cdf49a28fb89030ca05e2fd/users/dca0d7ea822a475aa7747b61d51fa23b/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.775548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b478d5-0733-43f2-8c90-33ea671d58d1 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:11.781261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef5390-1ccc-4dd2-9300-3eb2a0945548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2651c5c5cdf49a28fb89030ca05e2fd, user_id=dca0d7ea822a475aa7747b61d51fa23b, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:11.788992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:11.789640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.790013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.792535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef5390-1ccc-4dd2-9300-3eb2a0945548 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:11.794361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.819822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:11.820883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 823/1640] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:11 2022] PUT /identity/v3/projects/e2651c5c5cdf49a28fb89030ca05e2fd/users/dca0d7ea822a475aa7747b61d51fa23b/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:11.824912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:11.831549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e612d83-1f83-47ee-9a94-fd8b6bc77de0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:11.831869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e612d83-1f83-47ee-9a94-fd8b6bc77de0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:11.832147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e612d83-1f83-47ee-9a94-fd8b6bc77de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:11.858744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 818/1641] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:11.898758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5e612d83-1f83-47ee-9a94-fd8b6bc77de0 None None] MFA Rules not processed for user `dca0d7ea822a475aa7747b61d51fa23b`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:11.900552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5e612d83-1f83-47ee-9a94-fd8b6bc77de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:11.926332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:12.093256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 824/1642] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:12.333820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:12.351791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4186-f40e-4914-b5cf-7970272fe547 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:12.477389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:12.510041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:12.510696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:12.510989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:12.511283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:12.535180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:12.536997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:12.563404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 819/1643] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:12.784793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e612d83-1f83-47ee-9a94-fd8b6bc77de0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:12.796771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e612d83-1f83-47ee-9a94-fd8b6bc77de0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:12.805449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:12.807312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:12.807660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:12.807937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:12.906367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:12.907969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:12.932973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 825/1644] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:13.151736 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:13.166272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef072e2-e94b-4d39-9557-0bed69f686a8 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:13.183277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369c5c0-e8f1-41d9-afe1-09ced80b30a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:13.183935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369c5c0-e8f1-41d9-afe1-09ced80b30a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:13.188586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369c5c0-e8f1-41d9-afe1-09ced80b30a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:13.188924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369c5c0-e8f1-41d9-afe1-09ced80b30a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:13.342389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369c5c0-e8f1-41d9-afe1-09ced80b30a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:13.347614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369c5c0-e8f1-41d9-afe1-09ced80b30a7 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:13.389336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 820/1645] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:16.747729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:16.953277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cc5ff-d286-4651-81f6-b7e0153e9786 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:16.968880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:16.972625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:16.973044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:16.973456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] PATH_INFO: `/v3/users/c7bc901f44404ae48fb265243d8ffdcb` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:16.978399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bc901f44404ae48fb265243d8ffdcb)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:16.986177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:17.098750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bc901f44404ae48fb265243d8ffdcb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:17.100804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bc901f44404ae48fb265243d8ffdcb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:17.107287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:17.122430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bc901f44404ae48fb265243d8ffdcb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:17.124802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bc901f44404ae48fb265243d8ffdcb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:17.145877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bc901f44404ae48fb265243d8ffdcb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:17.191384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 826/1646] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:16 2022] DELETE /identity/v3/users/c7bc901f44404ae48fb265243d8ffdcb => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:17.196242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdc22-88c5-4388-8e3f-11b10c614f01 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:17.196772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdc22-88c5-4388-8e3f-11b10c614f01 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:17.196772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdc22-88c5-4388-8e3f-11b10c614f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:17.220626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-58bbdc22-88c5-4388-8e3f-11b10c614f01 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:17.222414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-58bbdc22-88c5-4388-8e3f-11b10c614f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:17.318948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 821/1647] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:17.345083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:17.347013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:17.472505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f27dc-2d2c-431c-8cd1-12d174677f4c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:17.478495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41437-0327-4157-b382-0ae2ac46cde9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:17.479307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41437-0327-4157-b382-0ae2ac46cde9 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:17.479620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41437-0327-4157-b382-0ae2ac46cde9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:17.480262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41437-0327-4157-b382-0ae2ac46cde9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:17.504071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41437-0327-4157-b382-0ae2ac46cde9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:17.507303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41437-0327-4157-b382-0ae2ac46cde9 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:17.523488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 827/1648] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:17.592038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:17.592380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:17.592599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:17.700468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] MFA Rules not processed for user `9643b0f10e8345889c181efbb1978237`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:17.703405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:17.953764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 822/1649] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:17.955731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:17.956202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:17.956639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.007050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] MFA Rules not processed for user `9643b0f10e8345889c181efbb1978237`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:18.008493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:18.032373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 828/1650] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:18.039738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:18.042003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:18.081298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:18.259801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5311e-737f-4873-bdc6-0cf3efed7eee None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:18.300337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0178df-8843-4478-8071-8841caacfd4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:18.301116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0178df-8843-4478-8071-8841caacfd4d service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:18.301116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0178df-8843-4478-8071-8841caacfd4d service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:18.301299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0178df-8843-4478-8071-8841caacfd4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.314731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:18.316713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:18.321668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0178df-8843-4478-8071-8841caacfd4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:18.323394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0178df-8843-4478-8071-8841caacfd4d service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:18.341226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258374f-fa8d-499e-8039-7859bce99bcf None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:18.346007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 823/1651] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:18.367118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:18.369131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:18.369614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:18.370092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] PATH_INFO: `/v3/projects/b6bddea5c8484dab9602d2eb375aafc1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.371683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bddea5c8484dab9602d2eb375aafc1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:18.374591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: WARNING py.warnings [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:38:18.374591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: warnings.warn(msg) Aug 22 05:38:18.374591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: Aug 22 05:38:18.376988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:18.539639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bddea5c8484dab9602d2eb375aafc1', 'request_id': 'req-455b7a0b-4c88-4db9-87df-f490f5bbc504'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:18.578937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bddea5c8484dab9602d2eb375aafc1', 'request_id': 'req-455b7a0b-4c88-4db9-87df-f490f5bbc504'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:18.631298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bddea5c8484dab9602d2eb375aafc1', 'request_id': 'req-455b7a0b-4c88-4db9-87df-f490f5bbc504'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:18.660618 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] The token cache is being invalidate because project b6bddea5c8484dab9602d2eb375aafc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:18.661272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:18.664299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-455b7a0b-4c88-4db9-87df-f490f5bbc504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:18.666430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 829/1652] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:18 2022] DELETE /identity/v3/projects/b6bddea5c8484dab9602d2eb375aafc1 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:18.672224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3794f-3afb-408f-a5d6-38f96951cc06 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:18.672516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3794f-3afb-408f-a5d6-38f96951cc06 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:18.672789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3794f-3afb-408f-a5d6-38f96951cc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.707362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-58a3794f-3afb-408f-a5d6-38f96951cc06 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:18.712407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-58a3794f-3afb-408f-a5d6-38f96951cc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:18.738163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf9dce-20c5-4311-917a-36e97bd79655 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:18.738661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf9dce-20c5-4311-917a-36e97bd79655 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:18.739090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf9dce-20c5-4311-917a-36e97bd79655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.792365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7ebf9dce-20c5-4311-917a-36e97bd79655 None None] MFA Rules not processed for user `904630cc2e60419d88afd9cfc1c5914d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:18.793964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7ebf9dce-20c5-4311-917a-36e97bd79655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:18.818306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 824/1653] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:18.827978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d687-051b-4686-8be0-721514ff5090 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:18.828282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d687-051b-4686-8be0-721514ff5090 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:18.828539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d687-051b-4686-8be0-721514ff5090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.853058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:18.867923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0564d687-051b-4686-8be0-721514ff5090 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:18.869198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0564d687-051b-4686-8be0-721514ff5090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:18.897147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 825/1654] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:18.898901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564d687-051b-4686-8be0-721514ff5090 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:18.905204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 830/1655] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:18.916267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57f6d-ffc1-48f8-a4b3-a00cbb192def None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:18.916637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57f6d-ffc1-48f8-a4b3-a00cbb192def None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:18.917886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c57f6d-ffc1-48f8-a4b3-a00cbb192def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.920709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564d687-051b-4686-8be0-721514ff5090 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:18.940171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:18.942201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:18.942965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:18.944184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:18.945364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:18.947166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:18.951154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:18.965527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-08c57f6d-ffc1-48f8-a4b3-a00cbb192def None None] MFA Rules not processed for user `904630cc2e60419d88afd9cfc1c5914d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:18.970362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-08c57f6d-ffc1-48f8-a4b3-a00cbb192def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:18.991918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 831/1656] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:18.993817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57f6d-ffc1-48f8-a4b3-a00cbb192def None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:18.996779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 826/1657] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:19.004125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:19.012497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:19.031959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c57f6d-ffc1-48f8-a4b3-a00cbb192def None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:19.045376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:19.046331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.046797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.047240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.048155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:19.058553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:19.109191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8382986-69c9-4269-bcef-791a9f335d86 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:19.118248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f4130-d8b9-4417-b2af-5c88065e7d9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:19.119019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f4130-d8b9-4417-b2af-5c88065e7d9d service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.119388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f4130-d8b9-4417-b2af-5c88065e7d9d service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.119730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f4130-d8b9-4417-b2af-5c88065e7d9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.121870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 832/1658] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:18 2022] POST /identity/v3/projects => generated 363 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:19.126127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:19.149343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f4130-d8b9-4417-b2af-5c88065e7d9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:19.151360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f4130-d8b9-4417-b2af-5c88065e7d9d service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:19.205521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 827/1659] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:19.302529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1e488-aae7-405a-880b-b5a12b9c1112 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.303185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1e488-aae7-405a-880b-b5a12b9c1112 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.303185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1e488-aae7-405a-880b-b5a12b9c1112 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.304502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 828/1660] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 22 05:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:38:19.318676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55773-6b0a-4971-ba8c-d067b36ace90 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.318959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55773-6b0a-4971-ba8c-d067b36ace90 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.319232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55773-6b0a-4971-ba8c-d067b36ace90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.330036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:19.364389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f887ff-9bde-48a6-9496-e470113c67ca admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:19.382688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:19.386900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.387270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.388058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.391316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:19.393708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:19.430691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-77f55773-6b0a-4971-ba8c-d067b36ace90 None None] MFA Rules not processed for user `0a5c594aafc14c1e86ffa9702f9be964`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:19.432341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-77f55773-6b0a-4971-ba8c-d067b36ace90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:19.453612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 833/1661] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:19 2022] POST /identity/v3/users => generated 282 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:19.473297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:19.492450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a3253-dd94-4eff-bb31-f4120297c32f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:19.505610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:19.505610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.505610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.505610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.506665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:19.506665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:19.521874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 834/1662] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:19.530665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:19.552822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9d9f5-1a7a-4d65-95b2-4a7dc09f6b1c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:19.559119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:19.560126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.560729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.561198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.562060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:19.565032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:19.569714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 829/1663] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:38:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:19.589829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 835/1664] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:19.595385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55773-6b0a-4971-ba8c-d067b36ace90 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:19.605247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:19.607482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:19.625733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55773-6b0a-4971-ba8c-d067b36ace90 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:19.633643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13751ad3-793b-47d8-b271-7a7f6570c466 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:19.638099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad72d-ff3d-4a17-a660-55873b1bacae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:19.639035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad72d-ff3d-4a17-a660-55873b1bacae service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.640193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad72d-ff3d-4a17-a660-55873b1bacae service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.640710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad72d-ff3d-4a17-a660-55873b1bacae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.642974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:19.643794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.644081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.644336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] PATH_INFO: `/v3/projects/cbcbfc7f52da426e8b468cd4fe71697d/users/342ac7cbf3e74a5a860be88e58acffe5/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.651082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad72d-ff3d-4a17-a660-55873b1bacae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:19.653876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad72d-ff3d-4a17-a660-55873b1bacae service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:19.665889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 836/1665] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:19.694707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbcbfc7f52da426e8b468cd4fe71697d, user_id=342ac7cbf3e74a5a860be88e58acffe5, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:19.697501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:19.725585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 830/1666] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:19 2022] PUT /identity/v3/projects/cbcbfc7f52da426e8b468cd4fe71697d/users/342ac7cbf3e74a5a860be88e58acffe5/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:19.741968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868643e-c1f2-45f1-a2e9-a1449bc3829e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:19.742711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868643e-c1f2-45f1-a2e9-a1449bc3829e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:19.742941 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868643e-c1f2-45f1-a2e9-a1449bc3829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:19.832813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7868643e-c1f2-45f1-a2e9-a1449bc3829e None None] MFA Rules not processed for user `342ac7cbf3e74a5a860be88e58acffe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:19.834091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7868643e-c1f2-45f1-a2e9-a1449bc3829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:19.904555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:19.927974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 837/1667] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:19.932725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:20.056969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fe71a-26a0-43e3-a410-abeaa98cf536 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:20.071429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:20.073388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.073671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.073943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.085783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:20.088020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:20.129804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479549-6054-41f8-a6fe-0859bf01f629 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.130814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479549-6054-41f8-a6fe-0859bf01f629 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.131311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479549-6054-41f8-a6fe-0859bf01f629 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.132985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 838/1668] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 22 05:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:38:20.138917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8391258-0012-444d-8aa1-317944632f61 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.139410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8391258-0012-444d-8aa1-317944632f61 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.139973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8391258-0012-444d-8aa1-317944632f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.164543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 831/1669] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:19 2022] POST /identity/v3/projects => generated 363 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:20.170503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:20.170688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b8391258-0012-444d-8aa1-317944632f61 None None] MFA Rules not processed for user `0a5c594aafc14c1e86ffa9702f9be964`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:20.172263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b8391258-0012-444d-8aa1-317944632f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:20.235193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 839/1670] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:38:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:20.326389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd1e0e-5b8a-460e-a058-4b5471743a39 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:20.331566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:20.332225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.332504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.332775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.333429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:20.336869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:20.347891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:20.368967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8391258-0012-444d-8aa1-317944632f61 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:20.401353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8391258-0012-444d-8aa1-317944632f61 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:20.420245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 832/1671] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:20 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:20.426912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:20.427223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:20.428214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.428839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.430184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.459649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:20.461478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:20.462383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76cc56-bf7d-46e3-b157-2ce858dd2f44 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:20.481804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:20.482246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.482531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.482803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.483463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:20.485122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:20.515142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 840/1672] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:20.534276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 833/1673] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:20.538243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:20.565275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35531-ee9b-4705-9467-130ddf627032 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:20.581375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:20.582133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.582474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.582828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] PATH_INFO: `/v3/projects/3bc06d814852466bac42032761ec263d/users/9cf91beaf8f84b67b42d8b28215c7dec/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.616873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc06d814852466bac42032761ec263d, user_id=9cf91beaf8f84b67b42d8b28215c7dec, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:20.619964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:20.650039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 841/1674] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:20 2022] PUT /identity/v3/projects/3bc06d814852466bac42032761ec263d/users/9cf91beaf8f84b67b42d8b28215c7dec/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:20.654147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:20.668957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49becda6-c576-496f-a4dd-224238aceaf9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:20.851086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9afd8c-8967-4dde-8414-6ddc3d5d090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:20.851901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9afd8c-8967-4dde-8414-6ddc3d5d090e admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.852370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9afd8c-8967-4dde-8414-6ddc3d5d090e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.852880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9afd8c-8967-4dde-8414-6ddc3d5d090e admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.854189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9afd8c-8967-4dde-8414-6ddc3d5d090e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:20.864901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9afd8c-8967-4dde-8414-6ddc3d5d090e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:20.910791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 834/1675] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:20.914970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:20.942291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c36b3-eafd-4368-b37a-9b8bc94e1631 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:20.981538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:20.990408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:20.990934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:20.991336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] PATH_INFO: `/v3/projects/3bc06d814852466bac42032761ec263d/users/9cf91beaf8f84b67b42d8b28215c7dec/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:20.993677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc06d814852466bac42032761ec263d, user_id=9cf91beaf8f84b67b42d8b28215c7dec, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:20.996698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:21.017813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 842/1676] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:20 2022] PUT /identity/v3/projects/3bc06d814852466bac42032761ec263d/users/9cf91beaf8f84b67b42d8b28215c7dec/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:21.030732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:21.031684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:21.031992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:21.105135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:21.107673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:21.128362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] MFA Rules not processed for user `9cf91beaf8f84b67b42d8b28215c7dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:21.130225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:21.396306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 835/1677] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:21.397821 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dbbfe-862c-4453-9a13-d2a830024f29 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:21.413197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:21.415798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:21.424226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b24e3-1ad4-40a3-ad0e-16d6901ac747 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:21.426060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b24e3-1ad4-40a3-ad0e-16d6901ac747 service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:21.426060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b24e3-1ad4-40a3-ad0e-16d6901ac747 service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:21.426060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b24e3-1ad4-40a3-ad0e-16d6901ac747 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:21.431493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a52b24e3-1ad4-40a3-ad0e-16d6901ac747 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:21.463559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b24e3-1ad4-40a3-ad0e-16d6901ac747 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:21.465469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b24e3-1ad4-40a3-ad0e-16d6901ac747 service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:21.487754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 843/1678] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:21.490993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:21.491284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:21.491556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:21.542495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d076777-6cb6-44d8-b041-c907780823d8 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:21.552995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db62cd6-bbee-431b-a91c-3fd1ea659570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:21.555968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db62cd6-bbee-431b-a91c-3fd1ea659570 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:21.556362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db62cd6-bbee-431b-a91c-3fd1ea659570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:21.557384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db62cd6-bbee-431b-a91c-3fd1ea659570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:21.575366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] MFA Rules not processed for user `8e68f520aaff495abfb4b7a5271936e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:21.588105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db62cd6-bbee-431b-a91c-3fd1ea659570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:21.588105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db62cd6-bbee-431b-a91c-3fd1ea659570 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:21.611729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:21.632989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 836/1679] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:21.634971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ddac-a459-4d24-a920-91d684595bf5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:21.637241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ddac-a459-4d24-a920-91d684595bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:21.639974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ddac-a459-4d24-a920-91d684595bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:21.656350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:21.687610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:21.688656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ad79ddac-a459-4d24-a920-91d684595bf5 None None] MFA Rules not processed for user `9643b0f10e8345889c181efbb1978237`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:21.690324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ad79ddac-a459-4d24-a920-91d684595bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:21.798808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 844/1680] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:21.816049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:21.818912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:21.969884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 837/1681] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:22.089599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae26133-3fa7-4b31-ab3b-7a518aa57148 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:22.099101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be34b17-3bd5-4083-87e0-0bf2ac09e3ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:22.100438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be34b17-3bd5-4083-87e0-0bf2ac09e3ff service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:22.100765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be34b17-3bd5-4083-87e0-0bf2ac09e3ff service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:22.101046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be34b17-3bd5-4083-87e0-0bf2ac09e3ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:22.112491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be34b17-3bd5-4083-87e0-0bf2ac09e3ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:22.126372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be34b17-3bd5-4083-87e0-0bf2ac09e3ff service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:22.148221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 845/1682] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:23.039364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79ddac-a459-4d24-a920-91d684595bf5 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:23.241746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79ddac-a459-4d24-a920-91d684595bf5 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:23.264545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:23.265244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:23.265619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:23.265774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] PATH_INFO: `/v3/users/342ac7cbf3e74a5a860be88e58acffe5` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:23.267021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] RBAC: Authorizing `identity:delete_user(user_id=342ac7cbf3e74a5a860be88e58acffe5)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:23.268091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: WARNING py.warnings [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 05:38:23.268091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: warnings.warn(msg) Aug 22 05:38:23.268091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: Aug 22 05:38:23.269424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:23.443064 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342ac7cbf3e74a5a860be88e58acffe5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:23.445918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342ac7cbf3e74a5a860be88e58acffe5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:23.448118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342ac7cbf3e74a5a860be88e58acffe5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:23.475968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342ac7cbf3e74a5a860be88e58acffe5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:23.513120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342ac7cbf3e74a5a860be88e58acffe5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:23.609615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 838/1683] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:23 2022] DELETE /identity/v3/users/342ac7cbf3e74a5a860be88e58acffe5 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:23.614470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:23.614972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:23.615644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:23.691502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:23.693503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:23.840932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 846/1684] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:23.856737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:23.864775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:23.946264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:23.996589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e90772-6a4b-4ff7-a8f5-b00a52be320b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:24.002138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:24.003468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:24.003903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:24.004277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:24.018907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:24.020789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:24.035488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 839/1685] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:24.595804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:24.597502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:24.615764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088c738-9830-4dcb-b4d8-ae85703b1ac3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:24.627781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:24.628462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:24.628743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:24.629041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] PATH_INFO: `/v3/projects/cbcbfc7f52da426e8b468cd4fe71697d` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:24.630209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbcbfc7f52da426e8b468cd4fe71697d)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:24.632757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:24.695890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbcbfc7f52da426e8b468cd4fe71697d', 'request_id': 'req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.697167 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:24.709873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbcbfc7f52da426e8b468cd4fe71697d', 'request_id': 'req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.720203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbcbfc7f52da426e8b468cd4fe71697d', 'request_id': 'req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.736354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] The token cache is being invalidate because project cbcbfc7f52da426e8b468cd4fe71697d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:24.737089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.737867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-531e4bbb-0f6b-4045-b6cd-8317e5b779f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.740287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 847/1686] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:24 2022] DELETE /identity/v3/projects/cbcbfc7f52da426e8b468cd4fe71697d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:24.744311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:24.804985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a6da1-bf7c-4dea-ac50-a14290ac6789 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:24.896261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:24.897151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:24.897818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:24.898295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] PATH_INFO: `/v3/users/9cf91beaf8f84b67b42d8b28215c7dec` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:24.899865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cf91beaf8f84b67b42d8b28215c7dec)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:24.904761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:24.983732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cf91beaf8f84b67b42d8b28215c7dec'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.984673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cf91beaf8f84b67b42d8b28215c7dec'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.985542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cf91beaf8f84b67b42d8b28215c7dec'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:24.997920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cf91beaf8f84b67b42d8b28215c7dec'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:25.009291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cf91beaf8f84b67b42d8b28215c7dec'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:25.031993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 840/1687] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:24 2022] DELETE /identity/v3/users/9cf91beaf8f84b67b42d8b28215c7dec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:25.096356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3b35-6e8c-41b8-b8fc-61e0544b741d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:25.096648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3b35-6e8c-41b8-b8fc-61e0544b741d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:25.097016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3b35-6e8c-41b8-b8fc-61e0544b741d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:25.123435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d13f3b35-6e8c-41b8-b8fc-61e0544b741d None None] MFA Rules not processed for user `8e68f520aaff495abfb4b7a5271936e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:25.124964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d13f3b35-6e8c-41b8-b8fc-61e0544b741d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:25.253308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 848/1688] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:25.556843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:25.562423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:25.668412 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:25.701614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd447cbd-5645-4f2d-8491-f738d24a1ac9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:25.710680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:25.711700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:25.712147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:25.712562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] PATH_INFO: `/v3/projects/3bc06d814852466bac42032761ec263d` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:25.714076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc06d814852466bac42032761ec263d)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:25.716472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:25.787869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc06d814852466bac42032761ec263d', 'request_id': 'req-ec1d8552-1307-49fc-ae19-204dcb7ab268'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:25.795395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc06d814852466bac42032761ec263d', 'request_id': 'req-ec1d8552-1307-49fc-ae19-204dcb7ab268'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:25.804382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc06d814852466bac42032761ec263d', 'request_id': 'req-ec1d8552-1307-49fc-ae19-204dcb7ab268'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:25.825126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] The token cache is being invalidate because project 3bc06d814852466bac42032761ec263d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:25.825746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:25.826844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ec1d8552-1307-49fc-ae19-204dcb7ab268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:25.829011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 841/1689] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:25 2022] DELETE /identity/v3/projects/3bc06d814852466bac42032761ec263d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:25.834316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe851d-5429-45ce-949e-6b9539448fa5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:25.834810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe851d-5429-45ce-949e-6b9539448fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:25.835119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe851d-5429-45ce-949e-6b9539448fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:25.865703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4afe851d-5429-45ce-949e-6b9539448fa5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:25.867509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4afe851d-5429-45ce-949e-6b9539448fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:25.989338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 849/1690] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:25.997078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed708d-ec0d-4dcc-9dff-e3eb6cf3d5c7 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:25.997627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed708d-ec0d-4dcc-9dff-e3eb6cf3d5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:25.998054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed708d-ec0d-4dcc-9dff-e3eb6cf3d5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.033617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-16ed708d-ec0d-4dcc-9dff-e3eb6cf3d5c7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:26.036322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-16ed708d-ec0d-4dcc-9dff-e3eb6cf3d5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:26.058199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 842/1691] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:26.063152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe851d-5429-45ce-949e-6b9539448fa5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:26.099331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe851d-5429-45ce-949e-6b9539448fa5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:26.116783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:26.117888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:26.119160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:26.119648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.120555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:26.122636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:26.124687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:26.182916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 850/1692] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:26.188626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed708d-ec0d-4dcc-9dff-e3eb6cf3d5c7 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:26.206391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed708d-ec0d-4dcc-9dff-e3eb6cf3d5c7 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:26.220355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:26.221148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:26.221776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:26.222054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.222746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:26.224840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:26.295125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 843/1693] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:26 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:26.295907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:26.316311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5cdd5-0374-43b6-94a2-29ab085ae990 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:26.479107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:26.480175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:26.480613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:26.481096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.482574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:26.486574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:26.537884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:26.548359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 851/1694] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:26 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:26.549373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:26.565254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218ff26-cb84-4ace-ab5e-291989cb6a14 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:26.577472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:26.578430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:26.578798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:26.579157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.580285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:26.582174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:26.619299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 844/1695] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:38:26.620481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:26.636255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6b5d-6fd8-4f89-95ee-957bec877149 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:26.649503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88af96-2e3c-48b5-b529-05a593e7c56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:26.650321 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88af96-2e3c-48b5-b529-05a593e7c56c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:26.650627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88af96-2e3c-48b5-b529-05a593e7c56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:26.651097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88af96-2e3c-48b5-b529-05a593e7c56c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.651822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88af96-2e3c-48b5-b529-05a593e7c56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:26.653571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88af96-2e3c-48b5-b529-05a593e7c56c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:26.682594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 852/1696] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:26.687328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:26.702408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570be5e-0966-4db1-8108-eaa391b36ea4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:26.708649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:26.709958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:26.710474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:26.710945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] PATH_INFO: `/v3/projects/76f2673913814b65870d97a33966444c/users/f519ba53264140098982d2ca6877a4b8/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.745117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f2673913814b65870d97a33966444c, user_id=f519ba53264140098982d2ca6877a4b8, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:26.748431 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:26.774957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 845/1697] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:26 2022] PUT /identity/v3/projects/76f2673913814b65870d97a33966444c/users/f519ba53264140098982d2ca6877a4b8/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:26.789668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b58d6-e15a-4fef-9c7b-35d63e6b2a1f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:26.790009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b58d6-e15a-4fef-9c7b-35d63e6b2a1f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:26.790399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b58d6-e15a-4fef-9c7b-35d63e6b2a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:26.842527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-304b58d6-e15a-4fef-9c7b-35d63e6b2a1f None None] MFA Rules not processed for user `f519ba53264140098982d2ca6877a4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:26.845128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-304b58d6-e15a-4fef-9c7b-35d63e6b2a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:26.946144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 853/1698] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:26.951093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:27.030189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9789-e1e7-41f9-88ae-02a1a20c3d93 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:27.035612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:27.036709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:27.037224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:27.037698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:27.038370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:27.041115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:27.067453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:27.076781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 846/1699] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:26 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:27.082512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b58d6-e15a-4fef-9c7b-35d63e6b2a1f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:27.099959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b58d6-e15a-4fef-9c7b-35d63e6b2a1f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:27.219170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:27.219937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:27.223727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:27.224314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:27.229680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:27.230522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:27.255093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 854/1700] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:27 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:27.259261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:27.282803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef54d-1fb5-4d84-97bd-5a5348630375 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:27.288430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:27.289050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:27.289334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:27.289600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:27.290480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:27.292354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:27.319943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 847/1701] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:27.323759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:27.341194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f91fb2-a774-4ffa-b72d-2886f836f8fb admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:27.354223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:27.355077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:27.355636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:27.356085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] PATH_INFO: `/v3/projects/bd6d9c79f4ca46e080b521f48b658297/users/e608cf02112548b2a4578dd9454e1f6c/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:27.376376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6d9c79f4ca46e080b521f48b658297, user_id=e608cf02112548b2a4578dd9454e1f6c, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:27.379151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:27.395209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 855/1702] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:27 2022] PUT /identity/v3/projects/bd6d9c79f4ca46e080b521f48b658297/users/e608cf02112548b2a4578dd9454e1f6c/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:27.399229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:27.410420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef244ad-3e48-4b27-9b24-a97da67aa5df admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:27.521710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6050a0-ca70-4f47-b511-497c5ce033e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:27.522619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6050a0-ca70-4f47-b511-497c5ce033e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:27.524489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6050a0-ca70-4f47-b511-497c5ce033e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:27.525171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6050a0-ca70-4f47-b511-497c5ce033e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:27.527925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6050a0-ca70-4f47-b511-497c5ce033e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:27.534806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6050a0-ca70-4f47-b511-497c5ce033e8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:27.570664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:27.571718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 848/1703] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:27.602208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f9248-0a22-47f8-b5e9-f12b5a571e86 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:27.618035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:27.618789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:27.619910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:27.620751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] PATH_INFO: `/v3/projects/bd6d9c79f4ca46e080b521f48b658297/users/e608cf02112548b2a4578dd9454e1f6c/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:27.624380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6d9c79f4ca46e080b521f48b658297, user_id=e608cf02112548b2a4578dd9454e1f6c, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:27.630749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:27.702542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 856/1704] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:27 2022] PUT /identity/v3/projects/bd6d9c79f4ca46e080b521f48b658297/users/e608cf02112548b2a4578dd9454e1f6c/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:27.715754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d64f0-f819-41a5-833a-ff18113d3034 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:27.716041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d64f0-f819-41a5-833a-ff18113d3034 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:27.716361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d64f0-f819-41a5-833a-ff18113d3034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:27.837176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5e2d64f0-f819-41a5-833a-ff18113d3034 None None] MFA Rules not processed for user `e608cf02112548b2a4578dd9454e1f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:27.846051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5e2d64f0-f819-41a5-833a-ff18113d3034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:28.040022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 849/1705] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:28.051872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:28.054347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:28.112572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:28.145973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb85c4-f5a2-4153-ad71-4ebf3ac61225 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:28.151065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:28.151818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:28.152107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:28.152582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:28.230596 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:28.232564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:28.249353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 857/1706] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:32.091471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978369-e88f-40b8-8027-234cc469aa50 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:32.091970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978369-e88f-40b8-8027-234cc469aa50 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:32.092042 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978369-e88f-40b8-8027-234cc469aa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:32.217624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-18978369-e88f-40b8-8027-234cc469aa50 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:32.219357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-18978369-e88f-40b8-8027-234cc469aa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:32.331270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:32.372545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 850/1707] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:38:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:32.503139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:32.504816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:32.613856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96690f-93d4-4cbb-916f-3016dc738a67 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:32.629595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:32.630325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:32.630672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:32.630943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:32.656454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:32.658824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:32.669897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 858/1708] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:38:32.879933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:32.880277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:32.880556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:32.947503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] MFA Rules not processed for user `0a5c594aafc14c1e86ffa9702f9be964`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:32.949645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:33.134617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 851/1709] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 22 05:38:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:33.150697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:33.152753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:33.178113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c69cd-d466-4c08-974c-4754d17b3ad1 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:33.184688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836e55d-5409-4372-9fbc-93deefc69e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:33.185640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e55d-5409-4372-9fbc-93deefc69e26 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:33.185869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e55d-5409-4372-9fbc-93deefc69e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:33.186257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e55d-5409-4372-9fbc-93deefc69e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:33.199908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836e55d-5409-4372-9fbc-93deefc69e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:33.201878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836e55d-5409-4372-9fbc-93deefc69e26 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:33.213116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 859/1710] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:34.249683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:34.252058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:34.405279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45170972-c184-4036-af01-8410e8ef8ae9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:34.418047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:34.419161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:34.419822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:34.420654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] PATH_INFO: `/v3/users/7395ad97ab494323a7f92ea53b3d910a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:34.423347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7395ad97ab494323a7f92ea53b3d910a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:34.428407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:34.563672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7395ad97ab494323a7f92ea53b3d910a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:34.564817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7395ad97ab494323a7f92ea53b3d910a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:34.565464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7395ad97ab494323a7f92ea53b3d910a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:34.578384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7395ad97ab494323a7f92ea53b3d910a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:34.603903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7395ad97ab494323a7f92ea53b3d910a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:34.632570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 852/1711] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:34 2022] DELETE /identity/v3/users/7395ad97ab494323a7f92ea53b3d910a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:34.638014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d739d3-cda8-44ef-83c6-b58ad3f236da None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:34.638570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d739d3-cda8-44ef-83c6-b58ad3f236da None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:34.638570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d739d3-cda8-44ef-83c6-b58ad3f236da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:34.661368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-83d739d3-cda8-44ef-83c6-b58ad3f236da None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:34.664057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-83d739d3-cda8-44ef-83c6-b58ad3f236da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:34.714995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:34.744728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 860/1712] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:34.789196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:34.790938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:34.822934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:34.971243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058eda-1a55-45d5-a73b-4d3bd1cf7806 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:34.975588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd1bd8-d84d-47bb-86b0-1039465cf7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:34.976418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1bd8-d84d-47bb-86b0-1039465cf7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:34.976725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1bd8-d84d-47bb-86b0-1039465cf7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:34.976992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1bd8-d84d-47bb-86b0-1039465cf7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.020889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd1bd8-d84d-47bb-86b0-1039465cf7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:35.022628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd1bd8-d84d-47bb-86b0-1039465cf7d3 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:35.065612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 853/1713] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:35.514495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd241-1084-47a7-8332-6ef3054883ea None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:35.515058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd241-1084-47a7-8332-6ef3054883ea None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:35.515365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd241-1084-47a7-8332-6ef3054883ea None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.517104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 861/1714] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 05:38:35.563621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:35.564666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:35.565317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.607154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:35.608434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:35.674917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dd241-1084-47a7-8332-6ef3054883ea None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:35.676914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d07dd241-1084-47a7-8332-6ef3054883ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:35.696912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dd241-1084-47a7-8332-6ef3054883ea None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:35.705855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:35.706606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:35.707046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:35.707469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] PATH_INFO: `/v3/projects/c37fd468d46f4348b13b002e26f2ee3a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.711707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37fd468d46f4348b13b002e26f2ee3a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:35.715243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:35.718605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 854/1715] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:38:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:35.734080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:35.735510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:35.757663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94f7c4-1ee0-42fb-92c4-eb1dca4b3112 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:35.765334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bb136-60f4-4b8d-8bd5-8858dbc35cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:35.766073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bb136-60f4-4b8d-8bd5-8858dbc35cd7 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:35.766361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bb136-60f4-4b8d-8bd5-8858dbc35cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:35.766635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bb136-60f4-4b8d-8bd5-8858dbc35cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.776753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bb136-60f4-4b8d-8bd5-8858dbc35cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:35.778568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bb136-60f4-4b8d-8bd5-8858dbc35cd7 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:35.780887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37fd468d46f4348b13b002e26f2ee3a', 'request_id': 'req-60d911d1-0ab2-4468-b3d5-8e7ff2932218'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:35.786602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37fd468d46f4348b13b002e26f2ee3a', 'request_id': 'req-60d911d1-0ab2-4468-b3d5-8e7ff2932218'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:35.789309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 855/1716] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:35.793015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37fd468d46f4348b13b002e26f2ee3a', 'request_id': 'req-60d911d1-0ab2-4468-b3d5-8e7ff2932218'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:35.806294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] The token cache is being invalidate because project c37fd468d46f4348b13b002e26f2ee3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:35.806585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:35.807309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-60d911d1-0ab2-4468-b3d5-8e7ff2932218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:35.809483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 862/1717] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:35 2022] DELETE /identity/v3/projects/c37fd468d46f4348b13b002e26f2ee3a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:35.814244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665a306-338e-44b2-93bc-7c23e7e359fa None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:35.814530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665a306-338e-44b2-93bc-7c23e7e359fa None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:35.814801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665a306-338e-44b2-93bc-7c23e7e359fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.833903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4665a306-338e-44b2-93bc-7c23e7e359fa None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:35.835092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4665a306-338e-44b2-93bc-7c23e7e359fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:35.898552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 856/1718] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:35.906701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca404f-08d6-4956-8e20-fc57bf84deab None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:35.908181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca404f-08d6-4956-8e20-fc57bf84deab None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:35.908460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca404f-08d6-4956-8e20-fc57bf84deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.929978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2cca404f-08d6-4956-8e20-fc57bf84deab None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:35.931504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2cca404f-08d6-4956-8e20-fc57bf84deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:35.940154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 863/1719] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:35.944934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665a306-338e-44b2-93bc-7c23e7e359fa None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:35.964075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665a306-338e-44b2-93bc-7c23e7e359fa None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:35.969302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:35.969940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:35.970322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:35.970600 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:35.971279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:35.971606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:35.973555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:35.987542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 857/1720] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:35.992002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca404f-08d6-4956-8e20-fc57bf84deab None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.002168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca404f-08d6-4956-8e20-fc57bf84deab None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.006756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.007365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.007685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.007958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.008597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.010507 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.055134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 864/1721] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:35 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:36.056839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.073393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38c608-54aa-4443-8738-891591e3c2a3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.104943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:36.164701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.165868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.166327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.166960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.168289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.171044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.199711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 858/1722] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:36 2022] POST /identity/v3/users => generated 303 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:36.203791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.216250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58944ee5-db75-4058-b833-540db2461ec3 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.235308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.236614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.236912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.237244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.237921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.239731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.264657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 865/1723] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:36.268605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.279819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2681474-c734-4ba3-b61c-f661a4143c3e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.287517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa0009-cd5c-4c1f-9861-d7f68e62fb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.288802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa0009-cd5c-4c1f-9861-d7f68e62fb7e admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.289297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa0009-cd5c-4c1f-9861-d7f68e62fb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.289772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa0009-cd5c-4c1f-9861-d7f68e62fb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.290689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa0009-cd5c-4c1f-9861-d7f68e62fb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.292539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa0009-cd5c-4c1f-9861-d7f68e62fb7e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.314856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 859/1724] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:36.319406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.332169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ea057-dad9-4fe6-832a-49b37016c672 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.337288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.338056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.338403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.338748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] PATH_INFO: `/v3/projects/fa59f61f203d4a6f91e4b632eef2c6ad/users/51640707eab1455a85117c3d1c29025e/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.368386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa59f61f203d4a6f91e4b632eef2c6ad, user_id=51640707eab1455a85117c3d1c29025e, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.374833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.396340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 866/1725] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:36 2022] PUT /identity/v3/projects/fa59f61f203d4a6f91e4b632eef2c6ad/users/51640707eab1455a85117c3d1c29025e/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:36.417871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573dc4-8830-419a-bad4-a2806e07d301 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.418326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573dc4-8830-419a-bad4-a2806e07d301 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.418671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573dc4-8830-419a-bad4-a2806e07d301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.455243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-d9573dc4-8830-419a-bad4-a2806e07d301 None None] MFA Rules not processed for user `51640707eab1455a85117c3d1c29025e`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:36.457229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d9573dc4-8830-419a-bad4-a2806e07d301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:36.555748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 860/1726] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:36.559956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.655433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26d422-1252-4a03-9d0e-8697a9fd1c0c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.661363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.662561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.663152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.663891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.664782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.667640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.676205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:36.726087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 867/1727] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:36 2022] POST /identity/v3/projects => generated 405 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:36.730009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573dc4-8830-419a-bad4-a2806e07d301 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.760852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573dc4-8830-419a-bad4-a2806e07d301 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.823112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.824158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.824445 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.824718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.825389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.827358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.854873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 861/1728] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:36 2022] POST /identity/v3/users => generated 309 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:36.856625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.872965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fbca1-ebda-4497-9556-9f03f0b6ce9d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.881679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.882569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.882945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.883289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.884259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.886792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.897136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 868/1729] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:36.901528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.912035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba557a5-50ef-43bd-bbc6-9af97b1f4b35 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:36.917011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:36.917679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:36.917960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:36.918231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] PATH_INFO: `/v3/projects/4cd58d87acae4bd4afb0525f5e6d2b3f/users/63dbd0715ca441faa7ed766bc22d07bd/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:36.935817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd58d87acae4bd4afb0525f5e6d2b3f, user_id=63dbd0715ca441faa7ed766bc22d07bd, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:36.938513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:36.940832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:36.953686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 862/1730] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:36 2022] PUT /identity/v3/projects/4cd58d87acae4bd4afb0525f5e6d2b3f/users/63dbd0715ca441faa7ed766bc22d07bd/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:36.958408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:36.974312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1107cbc-b763-42f8-83c8-6d30bcecd155 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:37.045550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e8ef0-bc60-4fc3-9fbf-7ffd8c57c699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:37.046167 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8ef0-bc60-4fc3-9fbf-7ffd8c57c699 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:37.046445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8ef0-bc60-4fc3-9fbf-7ffd8c57c699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:37.046715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8ef0-bc60-4fc3-9fbf-7ffd8c57c699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:37.047359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e8ef0-bc60-4fc3-9fbf-7ffd8c57c699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:37.049000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e8ef0-bc60-4fc3-9fbf-7ffd8c57c699 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:37.059362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 869/1731] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:37.063986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:37.075544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b7a48-ba50-400d-bab0-6858995c8ee8 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:37.081122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:37.081903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:37.082216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:37.082502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] PATH_INFO: `/v3/projects/4cd58d87acae4bd4afb0525f5e6d2b3f/users/63dbd0715ca441faa7ed766bc22d07bd/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:37.084951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd58d87acae4bd4afb0525f5e6d2b3f, user_id=63dbd0715ca441faa7ed766bc22d07bd, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:37.088174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:37.111100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 863/1732] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:37 2022] PUT /identity/v3/projects/4cd58d87acae4bd4afb0525f5e6d2b3f/users/63dbd0715ca441faa7ed766bc22d07bd/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:37.113485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501abbf-f1a9-45cd-9d41-50d45489de85 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:37.113775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501abbf-f1a9-45cd-9d41-50d45489de85 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:37.114055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501abbf-f1a9-45cd-9d41-50d45489de85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:37.145355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-0501abbf-f1a9-45cd-9d41-50d45489de85 None None] MFA Rules not processed for user `63dbd0715ca441faa7ed766bc22d07bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:37.146597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0501abbf-f1a9-45cd-9d41-50d45489de85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:37.242142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 870/1733] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:37 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:37.250176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:37.252131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:37.319194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8cfc7-b843-4125-bb7b-f8bca983ee5d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:37.322751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b9831-cfe2-4840-82a4-0f7687e5a15e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:37.323362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9831-cfe2-4840-82a4-0f7687e5a15e service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:37.323671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9831-cfe2-4840-82a4-0f7687e5a15e service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:37.324067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9831-cfe2-4840-82a4-0f7687e5a15e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:37.334028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b9831-cfe2-4840-82a4-0f7687e5a15e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:37.335513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b9831-cfe2-4840-82a4-0f7687e5a15e service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:37.349010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 864/1734] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:37 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:39.470001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501abbf-f1a9-45cd-9d41-50d45489de85 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:39.535459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:39.540368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501abbf-f1a9-45cd-9d41-50d45489de85 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:39.545592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:39.546263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:39.546539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:39.546808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] PATH_INFO: `/v3/users/51640707eab1455a85117c3d1c29025e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:39.547967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51640707eab1455a85117c3d1c29025e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:39.549898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:39.602909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51640707eab1455a85117c3d1c29025e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:39.603691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51640707eab1455a85117c3d1c29025e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:39.611670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51640707eab1455a85117c3d1c29025e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:39.612505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51640707eab1455a85117c3d1c29025e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:39.624649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51640707eab1455a85117c3d1c29025e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:39.645738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 871/1735] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:39 2022] DELETE /identity/v3/users/51640707eab1455a85117c3d1c29025e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:39.650539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:39.651061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:39.651271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:39.689741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:39.691468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:39.770256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 865/1736] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:39.784768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:39.787522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:39.866653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082db96a-7f53-4678-8759-19e9e9befd2e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:39.870712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:39.871729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:39.872165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:39.872591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:39.882965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:39.885431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:39.901907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 872/1737] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:40.383165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:40.384824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:40.399488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254f8a0-74e7-4e9d-a0cb-307c3a403bfd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:40.404414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:40.405486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:40.405651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:40.405651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] PATH_INFO: `/v3/projects/fa59f61f203d4a6f91e4b632eef2c6ad` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:40.407882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa59f61f203d4a6f91e4b632eef2c6ad)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:40.409997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:40.471568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa59f61f203d4a6f91e4b632eef2c6ad', 'request_id': 'req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.477816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa59f61f203d4a6f91e4b632eef2c6ad', 'request_id': 'req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.484423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa59f61f203d4a6f91e4b632eef2c6ad', 'request_id': 'req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.499322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] The token cache is being invalidate because project fa59f61f203d4a6f91e4b632eef2c6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:40.499802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.500498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.502520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 866/1738] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:40 2022] DELETE /identity/v3/projects/fa59f61f203d4a6f91e4b632eef2c6ad => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:40.506912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:40.518072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c14ebc-2878-47b0-ae67-f1159e268d0c service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:40.602997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:40.604116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:40.604490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:40.604766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] PATH_INFO: `/v3/users/63dbd0715ca441faa7ed766bc22d07bd` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:40.606500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63dbd0715ca441faa7ed766bc22d07bd)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:40.609619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:40.640822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:40.673282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63dbd0715ca441faa7ed766bc22d07bd'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.674268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63dbd0715ca441faa7ed766bc22d07bd'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.683494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63dbd0715ca441faa7ed766bc22d07bd'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.684337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63dbd0715ca441faa7ed766bc22d07bd'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.693031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63dbd0715ca441faa7ed766bc22d07bd'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:40.710687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 873/1739] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:40 2022] DELETE /identity/v3/users/63dbd0715ca441faa7ed766bc22d07bd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:40.994324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:40.995924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:41.011496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:41.169823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6e97f-5a76-4b31-ad40-7c5ea0fb46d9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:41.313510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34b750-0966-47f9-8f88-53ec2e7897dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:41.314583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34b750-0966-47f9-8f88-53ec2e7897dc service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:41.314864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34b750-0966-47f9-8f88-53ec2e7897dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:41.315373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34b750-0966-47f9-8f88-53ec2e7897dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:41.320335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8e34b750-0966-47f9-8f88-53ec2e7897dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:41.436432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:41.438252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:41.518647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34b750-0966-47f9-8f88-53ec2e7897dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:41.522782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34b750-0966-47f9-8f88-53ec2e7897dc service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:41.552824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 867/1740] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:41.601945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec676f3-0ff1-4c5c-a2a9-90cdd080c943 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:41.611167 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:41.611457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:41.611768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:41.612041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] PATH_INFO: `/v3/projects/4cd58d87acae4bd4afb0525f5e6d2b3f` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:41.613233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd58d87acae4bd4afb0525f5e6d2b3f)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:41.615187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:41.696960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd58d87acae4bd4afb0525f5e6d2b3f', 'request_id': 'req-282f16c9-0af2-4ed3-8987-ff5bab88589d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:41.704396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd58d87acae4bd4afb0525f5e6d2b3f', 'request_id': 'req-282f16c9-0af2-4ed3-8987-ff5bab88589d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:41.712085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd58d87acae4bd4afb0525f5e6d2b3f', 'request_id': 'req-282f16c9-0af2-4ed3-8987-ff5bab88589d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:41.756700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] The token cache is being invalidate because project 4cd58d87acae4bd4afb0525f5e6d2b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:41.757081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:41.771655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-282f16c9-0af2-4ed3-8987-ff5bab88589d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:41.771655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 874/1741] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:41 2022] DELETE /identity/v3/projects/4cd58d87acae4bd4afb0525f5e6d2b3f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:41.772202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:41.772202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:41.772202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:41.839516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:41.841115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:41.939510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 868/1742] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:41.948762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923ad9-80a2-4883-9804-0178f2df6d6b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:41.949021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923ad9-80a2-4883-9804-0178f2df6d6b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:41.949299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923ad9-80a2-4883-9804-0178f2df6d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:41.988773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-54923ad9-80a2-4883-9804-0178f2df6d6b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:41.990275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-54923ad9-80a2-4883-9804-0178f2df6d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:41.991802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:41.993773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.012807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 875/1743] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:42.014647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923ad9-80a2-4883-9804-0178f2df6d6b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.015472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f533c-c081-4d15-b348-107f9f234d7e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.032620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923ad9-80a2-4883-9804-0178f2df6d6b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.035123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.036234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.036520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.036800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] PATH_INFO: `/v3/users/416ec4526eb244de813a87210c850da4` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.038890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.039481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=416ec4526eb244de813a87210c850da4)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.039866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.040222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.040567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.041369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:42.041626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.041828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.043736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.048228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:42.062682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 876/1744] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:38:42.069249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.085159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74c067-9255-41c5-ab5e-8a75350fc887 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.090663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.098938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.099425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.099907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.100441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.102726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.110047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:42.163711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 877/1745] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:42 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:42.163937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416ec4526eb244de813a87210c850da4'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.163937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416ec4526eb244de813a87210c850da4'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.163937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416ec4526eb244de813a87210c850da4'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.166914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.170786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.176188 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416ec4526eb244de813a87210c850da4'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.188016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e5a31a50-ada4-4713-b2ee-16fb835b90bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416ec4526eb244de813a87210c850da4'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.227238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 869/1746] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:41 2022] DELETE /identity/v3/users/416ec4526eb244de813a87210c850da4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:42.231695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.232128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.232405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.254489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:42.254489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.255251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf2a0-0a38-45f6-98ff-bdbdc03284d6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.347140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.349816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.351851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.352393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.354930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.358387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.367400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 870/1747] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:42.377585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.379669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.405271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 878/1748] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:42 2022] POST /identity/v3/users => generated 282 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:42.409963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.433109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd95cb0-6924-4010-989a-b10e2a596cdb admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.452117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.452806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.453026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.453319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.453972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.455601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.470644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 879/1749] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:42.472648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.487414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857ab38-5f14-457f-9111-0fa9916cc16c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.492017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.492732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.493013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.493285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.496995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d74e7-7039-4f20-850f-d8b4a4a5f9cb admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.505202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.508171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.509741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f4512-f2bb-4b52-8ee2-1dd1df819d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.512249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f4512-f2bb-4b52-8ee2-1dd1df819d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.512600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f4512-f2bb-4b52-8ee2-1dd1df819d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.512975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f4512-f2bb-4b52-8ee2-1dd1df819d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.514338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f4512-f2bb-4b52-8ee2-1dd1df819d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.518310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f4512-f2bb-4b52-8ee2-1dd1df819d8b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.543404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 871/1750] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:42.548382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 880/1751] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:42.552465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.563013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61b2c9-bd8e-4cfe-8ea6-4d539cd3c432 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.567647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.568318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.568595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.569086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] PATH_INFO: `/v3/projects/d9f882b621c44f928493f9afda0b9248/users/0eaaf3b575944a8c98e2311b92e216be/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.585730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f882b621c44f928493f9afda0b9248, user_id=0eaaf3b575944a8c98e2311b92e216be, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.588490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.605875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 872/1752] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:42 2022] PUT /identity/v3/projects/d9f882b621c44f928493f9afda0b9248/users/0eaaf3b575944a8c98e2311b92e216be/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:42.618732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19ba3d-0436-476e-a6f5-dd52b3727bef None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.619350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19ba3d-0436-476e-a6f5-dd52b3727bef None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.619350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19ba3d-0436-476e-a6f5-dd52b3727bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.648930 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.650980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.654390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-1d19ba3d-0436-476e-a6f5-dd52b3727bef None None] MFA Rules not processed for user `0eaaf3b575944a8c98e2311b92e216be`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:42.656389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1d19ba3d-0436-476e-a6f5-dd52b3727bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.659477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:42.732143 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074ff9d-5ab6-48fd-a9a9-d177bf80b97a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.736686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:42.738458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.739120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.739405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.739715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] PATH_INFO: `/v3/users/6c1bf81b1ab445408ea39d621874e274` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.740911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1bf81b1ab445408ea39d621874e274)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.742850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.759865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 881/1753] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:42.765078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19ba3d-0436-476e-a6f5-dd52b3727bef None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.778403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19ba3d-0436-476e-a6f5-dd52b3727bef None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:42.784383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:42.784817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.785095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.785364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.786008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:42.788201 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:42.816803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1bf81b1ab445408ea39d621874e274'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.817996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1bf81b1ab445408ea39d621874e274'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.818787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1bf81b1ab445408ea39d621874e274'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.830500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1bf81b1ab445408ea39d621874e274'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.831740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 882/1754] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:42 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:42.835039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:42.837037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.839899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fc826c8a-2a29-4a3e-82bc-dc000f85df1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1bf81b1ab445408ea39d621874e274'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:42.866219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 873/1755] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:42 2022] DELETE /identity/v3/users/6c1bf81b1ab445408ea39d621874e274 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:42.883762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:42.884567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:42.885834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:42.918263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:42.919929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:42.965255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391593d-6617-4b3e-9759-84277ef494fb admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.012038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 874/1756] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:43.012038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.014269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.037678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff776d54-0faa-449b-be5b-1b3594c4ac3a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.051593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.052422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.052705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.052974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] PATH_INFO: `/v3/projects/d6d9cabf2ed442e7b589b5aff223ef1b` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.061532 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6d9cabf2ed442e7b589b5aff223ef1b)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.064671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.091431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.092125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.092469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.092827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.093837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.095813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.111561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:43.142793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 883/1757] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:42 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:43.144204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.147414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.193168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d9cabf2ed442e7b589b5aff223ef1b', 'request_id': 'req-a8f62d83-6c06-40ec-9ffa-194a319436c0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:43.205207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d9cabf2ed442e7b589b5aff223ef1b', 'request_id': 'req-a8f62d83-6c06-40ec-9ffa-194a319436c0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:43.217121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d9cabf2ed442e7b589b5aff223ef1b', 'request_id': 'req-a8f62d83-6c06-40ec-9ffa-194a319436c0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:43.242477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] The token cache is being invalidate because project d6d9cabf2ed442e7b589b5aff223ef1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:43.243019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:43.243758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:43.250766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 875/1758] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:43 2022] DELETE /identity/v3/projects/d6d9cabf2ed442e7b589b5aff223ef1b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:43.250766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.253540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.283648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82266-9ce6-4ec9-95e0-18a39c9bbf98 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.319000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:43.371893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.372636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.373066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.373411 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.376138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.382388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f62d83-6c06-40ec-9ffa-194a319436c0 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.394113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.395234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816b0ef-c206-4350-ae7b-c03a9a32ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.395913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816b0ef-c206-4350-ae7b-c03a9a32ebbf admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.396086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.396200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816b0ef-c206-4350-ae7b-c03a9a32ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.396447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816b0ef-c206-4350-ae7b-c03a9a32ebbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.397112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816b0ef-c206-4350-ae7b-c03a9a32ebbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.398730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816b0ef-c206-4350-ae7b-c03a9a32ebbf admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.410624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 876/1759] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:43.412242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf4e59-af4e-473a-88cc-72ed9a6b40f9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.414733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf4e59-af4e-473a-88cc-72ed9a6b40f9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.415308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf4e59-af4e-473a-88cc-72ed9a6b40f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.419023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 884/1760] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:43.425272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.442836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a6c46-9e90-4f24-b18b-3ee71c5637cc service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.449598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.450381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.450729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.451073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] PATH_INFO: `/v3/projects/5c1c39ae6843448781cbbbfd5a0e5fdd/users/0d96ca77475c44559746e312b3469cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.464605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-eabf4e59-af4e-473a-88cc-72ed9a6b40f9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:43.465842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-eabf4e59-af4e-473a-88cc-72ed9a6b40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.469660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c1c39ae6843448781cbbbfd5a0e5fdd, user_id=0d96ca77475c44559746e312b3469cbf, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.472897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.476252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 877/1761] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:43.485222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7de4-f51b-4db3-a812-93b3f09e3dbd None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.485558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7de4-f51b-4db3-a812-93b3f09e3dbd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.485833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7de4-f51b-4db3-a812-93b3f09e3dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.498016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 885/1762] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:43 2022] PUT /identity/v3/projects/5c1c39ae6843448781cbbbfd5a0e5fdd/users/0d96ca77475c44559746e312b3469cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:43.503503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.530879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-944d7de4-f51b-4db3-a812-93b3f09e3dbd None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:43.534961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-944d7de4-f51b-4db3-a812-93b3f09e3dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.674027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d3d5-4628-4a8b-919a-bf2844c3f952 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.678985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 878/1763] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:43.682091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7de4-f51b-4db3-a812-93b3f09e3dbd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.684962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.685905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.686625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.687219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.688283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.690624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.694457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7de4-f51b-4db3-a812-93b3f09e3dbd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.700123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.700903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.701186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.701461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.705249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 886/1764] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:43.708501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:43.710063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.710983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.715214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.723390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950233f5-254a-4c4d-ad29-465d4abb8d9d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.729196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3a66f-16b9-4de4-991b-6bac1de99210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.729879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3a66f-16b9-4de4-991b-6bac1de99210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.730156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3a66f-16b9-4de4-991b-6bac1de99210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.730430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3a66f-16b9-4de4-991b-6bac1de99210 admin admin] PATH_INFO: `/v3/projects/5c1c39ae6843448781cbbbfd5a0e5fdd/users/0d96ca77475c44559746e312b3469cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.732514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3a66f-16b9-4de4-991b-6bac1de99210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c1c39ae6843448781cbbbfd5a0e5fdd, user_id=0d96ca77475c44559746e312b3469cbf, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.734356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 879/1765] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:43.735262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.735516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3a66f-16b9-4de4-991b-6bac1de99210 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.757178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 887/1766] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:43 2022] PUT /identity/v3/projects/5c1c39ae6843448781cbbbfd5a0e5fdd/users/0d96ca77475c44559746e312b3469cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:43.766779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a527bb-a0cd-4ed3-ae62-164e796df770 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.779988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.780642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.781180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.799978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:43.816107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] MFA Rules not processed for user `0d96ca77475c44559746e312b3469cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:43.818798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.843882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.844517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.844801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.845125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.845774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.847796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:43.877912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 880/1767] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:43 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:43.884127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.905782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 888/1768] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:43.906853 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:43.925023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:43.951073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e165f55-5a88-4e44-a0a5-6c687469ff98 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.956896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:43.976688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb0d14-287f-4dfb-8f1a-a4620bd3ce2f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:43.981464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:43.982080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:43.982359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:43.982631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:43.983371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:43.985366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.012205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 881/1769] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:43 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:44.013275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.015599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.042540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.043358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.043670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.043943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] PATH_INFO: `/v3/projects/c9f504dcd9e447c3b147c90f424544f7` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.045088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9f504dcd9e447c3b147c90f424544f7)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.047422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.076982 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c605ecb-b95f-4450-80ce-1bcd5a5e88c1 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.081196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.081820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.082099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.082364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.096116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9f504dcd9e447c3b147c90f424544f7', 'request_id': 'req-e3196e11-7434-479a-ad29-b916763d95e7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.101773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9f504dcd9e447c3b147c90f424544f7', 'request_id': 'req-e3196e11-7434-479a-ad29-b916763d95e7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.109956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9f504dcd9e447c3b147c90f424544f7', 'request_id': 'req-e3196e11-7434-479a-ad29-b916763d95e7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.125193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] The token cache is being invalidate because project c9f504dcd9e447c3b147c90f424544f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:44.125494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.126171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.130976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 889/1770] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:43 2022] DELETE /identity/v3/projects/c9f504dcd9e447c3b147c90f424544f7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:44.131852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.133216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.138152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:44.151429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.153839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.166247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 882/1771] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:44.167205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.179108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4490c1-29f6-4388-9512-05663a176387 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.228281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3196e11-7434-479a-ad29-b916763d95e7 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.245534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.246256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.246608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.247077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.247775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.250219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.268733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 890/1772] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:44.272789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.276985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.277774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.278040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.278491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] PATH_INFO: `/v3/users/00f29ca575df4ba9b489a0a52a9e7820` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.279519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f29ca575df4ba9b489a0a52a9e7820)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.282068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.284260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d355ae-7fd7-49d4-af5f-98362e284517 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.305590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.306642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.306642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.306918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.307588 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.309307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.321156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 891/1773] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:38:44.321989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.324092 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.348822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f29ca575df4ba9b489a0a52a9e7820'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.362832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f29ca575df4ba9b489a0a52a9e7820'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.362832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f29ca575df4ba9b489a0a52a9e7820'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.371065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:44.375319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f29ca575df4ba9b489a0a52a9e7820'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.390230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f29ca575df4ba9b489a0a52a9e7820'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.406735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6b51-2d03-4765-ad66-56c46d518d7a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.410784 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b026ced-f2e2-4ec7-8c96-be17017c5f5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.411484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b026ced-f2e2-4ec7-8c96-be17017c5f5a service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.411911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b026ced-f2e2-4ec7-8c96-be17017c5f5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.412313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b026ced-f2e2-4ec7-8c96-be17017c5f5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.412833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 883/1774] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:44 2022] DELETE /identity/v3/users/00f29ca575df4ba9b489a0a52a9e7820 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:44.415806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.416670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.428119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9b026ced-f2e2-4ec7-8c96-be17017c5f5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.494333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359555d0-7e87-47e4-9e85-d1e45927766a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.500212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.501071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.501597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.502011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] PATH_INFO: `/v3/projects/bac526eefb4545cab4907d935546afa3/users/2cfd701960a742399ace192c5c0258e9/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.526158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:44.526378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b026ced-f2e2-4ec7-8c96-be17017c5f5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.528088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b026ced-f2e2-4ec7-8c96-be17017c5f5a service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.528405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac526eefb4545cab4907d935546afa3, user_id=2cfd701960a742399ace192c5c0258e9, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.531393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.538126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 892/1775] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:44.557030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 884/1776] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:44 2022] PUT /identity/v3/projects/bac526eefb4545cab4907d935546afa3/users/2cfd701960a742399ace192c5c0258e9/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:44.560825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.561149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.561431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.593705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] MFA Rules not processed for user `2cfd701960a742399ace192c5c0258e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:44.595169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.670598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 893/1777] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:44.674761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.751068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9441d-e5b5-494f-bb18-08cbb9bf367c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.756052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.756765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.756997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.757469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.757469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.758264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.758965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.760512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.776087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beef0b7-025d-47ee-94f3-2a9860a9ec77 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.782443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.783336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.783647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.784104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] PATH_INFO: `/v3/projects/ac81afde9297428a8317b1bdf234f7d9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.785735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac81afde9297428a8317b1bdf234f7d9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.788458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:44.799680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 885/1778] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:44 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:44.802641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:44.855464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac81afde9297428a8317b1bdf234f7d9', 'request_id': 'req-3064d2ab-e953-45d5-8a61-72f3b8d85c98'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.864127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac81afde9297428a8317b1bdf234f7d9', 'request_id': 'req-3064d2ab-e953-45d5-8a61-72f3b8d85c98'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.871529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac81afde9297428a8317b1bdf234f7d9', 'request_id': 'req-3064d2ab-e953-45d5-8a61-72f3b8d85c98'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.873034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:44.879786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1d555-c300-4b4e-8821-d96e297f19d7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:44.888670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] The token cache is being invalidate because project ac81afde9297428a8317b1bdf234f7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:44.888967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.889698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3064d2ab-e953-45d5-8a61-72f3b8d85c98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:44.891687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 894/1779] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:44 2022] DELETE /identity/v3/projects/ac81afde9297428a8317b1bdf234f7d9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:44.896170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721d80-0b74-45e1-aab2-93c32f68607b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.897897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721d80-0b74-45e1-aab2-93c32f68607b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.898470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721d80-0b74-45e1-aab2-93c32f68607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.925356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4a721d80-0b74-45e1-aab2-93c32f68607b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:44.926376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4a721d80-0b74-45e1-aab2-93c32f68607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:44.970589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:44.971450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:44.972408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:44.972690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:44.973334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:44.975626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.011786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 886/1780] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:44 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:45.013161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.015601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:45.020858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 895/1781] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:45.027720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bf5ea-772b-4e61-baee-8305533f7aba admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.030058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98540b38-34cf-4393-b942-28324765edc1 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.030349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98540b38-34cf-4393-b942-28324765edc1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.030749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98540b38-34cf-4393-b942-28324765edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.033765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.034387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.034665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.034937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.035619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.037223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.048039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 887/1782] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:38:45.050854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-98540b38-34cf-4393-b942-28324765edc1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:45.052238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.052390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-98540b38-34cf-4393-b942-28324765edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:45.057280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:45.063803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c999-d802-4893-8926-b5a38e33d0da admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.067040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 896/1783] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:45.071492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98540b38-34cf-4393-b942-28324765edc1 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.073684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.078765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.080067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.080513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] PATH_INFO: `/v3/projects/cd2693b5ab5c451798cac69ca7a2e679/users/b8c1bc3cc515493993daffbdbf15ee20/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.081895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98540b38-34cf-4393-b942-28324765edc1 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.086798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.087561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.087987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.088602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.089384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:45.089814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.091768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.102710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2693b5ab5c451798cac69ca7a2e679, user_id=b8c1bc3cc515493993daffbdbf15ee20, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.106741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 897/1784] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:38:45.108860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.111304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.125372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439bb4e-2e98-4be8-9fcc-920a4f5c37cb admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.129102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 888/1785] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:45 2022] PUT /identity/v3/projects/cd2693b5ab5c451798cac69ca7a2e679/users/b8c1bc3cc515493993daffbdbf15ee20/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:45.130205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.203791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.206178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.208279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.208829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.209612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.212105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.216435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434cc9-939c-4412-b4d3-0c343093cebd admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.222358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.223005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.223281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.223547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.224769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.226450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.237078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 889/1786] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:45.241455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.245128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 898/1787] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:45 2022] POST /identity/v3/projects => generated 355 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:45.249376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.254161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38285ba7-a7e7-47c5-8c73-eab2cf12ce47 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.322173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:45.326822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb35b35-eca2-4beb-80df-848aaad990bd admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.332891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2ffc7-8214-4c49-905c-ab34e07b21d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.333662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2ffc7-8214-4c49-905c-ab34e07b21d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.334007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.334138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2ffc7-8214-4c49-905c-ab34e07b21d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.334476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2ffc7-8214-4c49-905c-ab34e07b21d9 admin admin] PATH_INFO: `/v3/projects/cd2693b5ab5c451798cac69ca7a2e679/users/b8c1bc3cc515493993daffbdbf15ee20/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.334800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.335172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.335854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.336583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.337477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2ffc7-8214-4c49-905c-ab34e07b21d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2693b5ab5c451798cac69ca7a2e679, user_id=b8c1bc3cc515493993daffbdbf15ee20, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.339017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.340362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2ffc7-8214-4c49-905c-ab34e07b21d9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.352948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 890/1788] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:45 2022] PUT /identity/v3/projects/cd2693b5ab5c451798cac69ca7a2e679/users/b8c1bc3cc515493993daffbdbf15ee20/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:45.365184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.365184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.365184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.385818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 899/1789] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:45 2022] POST /identity/v3/users => generated 278 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:38:45.390157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.403842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141891-e7de-41e6-b905-16df0f3b3e4b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.419438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] MFA Rules not processed for user `b8c1bc3cc515493993daffbdbf15ee20`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:45.421824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:45.539968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.540837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.541220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.542317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.543125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.546040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.572194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 900/1790] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:45.577866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.591905 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:45.598508 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e062e-a1a8-4a3f-a886-850600ef2347 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.604893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.605794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.606309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.606688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.607425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.609633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.634235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 901/1791] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:45.634235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.636221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 891/1792] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:45.648051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f63fd-d5f1-4e67-95a1-4138cef7e05d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.653377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fd1d3-343a-43c8-a691-967a99bdec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.654438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fd1d3-343a-43c8-a691-967a99bdec37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.654956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fd1d3-343a-43c8-a691-967a99bdec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.655534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fd1d3-343a-43c8-a691-967a99bdec37 admin admin] PATH_INFO: `/v3/projects/79318ba980164cc8bd4e516cd55886c6/users/2f70a12b43884d92abfbaf9167e35c88/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.659370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.661282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:45.690877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fd1d3-343a-43c8-a691-967a99bdec37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79318ba980164cc8bd4e516cd55886c6, user_id=2f70a12b43884d92abfbaf9167e35c88, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.695478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fd1d3-343a-43c8-a691-967a99bdec37 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.724087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 902/1793] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:45 2022] PUT /identity/v3/projects/79318ba980164cc8bd4e516cd55886c6/users/2f70a12b43884d92abfbaf9167e35c88/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:45.730346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05841181-104f-4aa6-a8a3-c9f834b9e16a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.730346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05841181-104f-4aa6-a8a3-c9f834b9e16a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.730346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05841181-104f-4aa6-a8a3-c9f834b9e16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.765731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-05841181-104f-4aa6-a8a3-c9f834b9e16a None None] MFA Rules not processed for user `2f70a12b43884d92abfbaf9167e35c88`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:45.768209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-05841181-104f-4aa6-a8a3-c9f834b9e16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:45.801593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85d56-3ab5-4bc9-a710-8def8ce57a79 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.872969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 903/1794] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:45.874999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05841181-104f-4aa6-a8a3-c9f834b9e16a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:45.879706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.880453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.880970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.881358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.928516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:45.970833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05841181-104f-4aa6-a8a3-c9f834b9e16a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:45.986029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:45.987720 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:45.990456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:45.991216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:45.991858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:45.992231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:45.993891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.004879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.007749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 892/1795] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:46.054147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 904/1796] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:45 2022] POST /identity/v3/projects => generated 357 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:46.057897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:46.085547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d117758-b713-4140-9065-5b0d5ae32e45 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:46.185735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:46.186568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.186914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.187253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.187986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.192577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.208456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:46.238377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 893/1797] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:46 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:46.246706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:46.262638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f64f7-dcb0-4426-9108-639338f7eee4 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:46.269934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:46.271174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.271524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.271892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.272625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.274535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.287665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 905/1798] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:46.300928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:46.303169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11f847-95a8-4001-babc-2f52cc154210 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:46.308158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:46.308834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.309111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.309377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] PATH_INFO: `/v3/projects/da9b2a0c65bd4e7697dafc9059973372/users/5d0652150115447ab08449d61bf05389/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.325224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9b2a0c65bd4e7697dafc9059973372, user_id=5d0652150115447ab08449d61bf05389, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.327984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.343693 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 894/1799] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:46 2022] PUT /identity/v3/projects/da9b2a0c65bd4e7697dafc9059973372/users/5d0652150115447ab08449d61bf05389/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:46.347802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:46.358579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75321c-9134-4e6f-8bd5-ddced315b77b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:46.416757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5960f-f8f6-41ac-949f-ebae4d2658f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:46.417544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5960f-f8f6-41ac-949f-ebae4d2658f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.417827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5960f-f8f6-41ac-949f-ebae4d2658f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.418100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5960f-f8f6-41ac-949f-ebae4d2658f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.418760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5960f-f8f6-41ac-949f-ebae4d2658f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.420523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5960f-f8f6-41ac-949f-ebae4d2658f3 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.431763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 906/1800] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:38:46.435267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:46.457044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d076ceb-8883-45a9-a3d0-ecd34dd2dc8b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:46.461968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:46.462647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.462926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.463316 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] PATH_INFO: `/v3/projects/da9b2a0c65bd4e7697dafc9059973372/users/5d0652150115447ab08449d61bf05389/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.465266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9b2a0c65bd4e7697dafc9059973372, user_id=5d0652150115447ab08449d61bf05389, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.468037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.485123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 895/1801] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:46 2022] PUT /identity/v3/projects/da9b2a0c65bd4e7697dafc9059973372/users/5d0652150115447ab08449d61bf05389/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:46.491898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.492246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.492447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.523703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] MFA Rules not processed for user `5d0652150115447ab08449d61bf05389`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:46.524915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:46.608760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 907/1802] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:46.621674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:46.682847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf8aed-c823-4fa2-8bb1-d07ded541589 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:46.686858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:46.687781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.688054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.688318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.699705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.701655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.715874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 896/1803] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:46.809195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:46.810665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:46.825748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018d84-023a-4715-a08c-c231311c4fbd None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:46.831244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da832f49-9859-4f1f-9c57-0198dd8f5f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:46.832039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da832f49-9859-4f1f-9c57-0198dd8f5f1b service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:46.832496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da832f49-9859-4f1f-9c57-0198dd8f5f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:46.832863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da832f49-9859-4f1f-9c57-0198dd8f5f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:46.888873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:46.922648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da832f49-9859-4f1f-9c57-0198dd8f5f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:46.924275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da832f49-9859-4f1f-9c57-0198dd8f5f1b service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:46.935879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 908/1804] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:47.868798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:47.949105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa94043-d28d-484f-93b9-865961760614 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:47.955552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:47.956521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:47.956980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:47.957402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] PATH_INFO: `/v3/users/2cfd701960a742399ace192c5c0258e9` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:47.958682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cfd701960a742399ace192c5c0258e9)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:47.960671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:48.029418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cfd701960a742399ace192c5c0258e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.030289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cfd701960a742399ace192c5c0258e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.030900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cfd701960a742399ace192c5c0258e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.040048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cfd701960a742399ace192c5c0258e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.042359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:48.048538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cfd701960a742399ace192c5c0258e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.073846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 897/1805] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:47 2022] DELETE /identity/v3/users/2cfd701960a742399ace192c5c0258e9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:48.079450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:48.080039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:48.080039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:48.102989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:48.104177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:48.194241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 909/1806] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:48.205535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:48.207166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:48.272439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f4ee3-c05c-4900-b8d0-f96e8d2dc879 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:48.276504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:48.277127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:48.277399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:48.277665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:48.290864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:48.292362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:48.302385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 898/1807] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:48.603317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:48.605116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:48.629527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b2b58-cabd-49dd-9ea5-93699acbcf42 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:48.641775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:48.642465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:48.642745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:48.643015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] PATH_INFO: `/v3/projects/bac526eefb4545cab4907d935546afa3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:48.644310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac526eefb4545cab4907d935546afa3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:48.646261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:48.697611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac526eefb4545cab4907d935546afa3', 'request_id': 'req-31b82d36-174d-43e1-8203-a944e1f35e88'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.703171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac526eefb4545cab4907d935546afa3', 'request_id': 'req-31b82d36-174d-43e1-8203-a944e1f35e88'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.709822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac526eefb4545cab4907d935546afa3', 'request_id': 'req-31b82d36-174d-43e1-8203-a944e1f35e88'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.725968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] The token cache is being invalidate because project bac526eefb4545cab4907d935546afa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:48.726371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.727086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.728687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 910/1808] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:48 2022] DELETE /identity/v3/projects/bac526eefb4545cab4907d935546afa3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:48.732918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:48.742618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734000d-c556-438a-ad72-dca6edcd43c6 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:48.801034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:48.801767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:48.802051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:48.802323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] PATH_INFO: `/v3/users/b8c1bc3cc515493993daffbdbf15ee20` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:48.803468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c1bc3cc515493993daffbdbf15ee20)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:48.805427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:48.855674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c1bc3cc515493993daffbdbf15ee20'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.856413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c1bc3cc515493993daffbdbf15ee20'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.857028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c1bc3cc515493993daffbdbf15ee20'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.866744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c1bc3cc515493993daffbdbf15ee20'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.876282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d92a9087-cdb2-4889-9ca4-8f17a994f469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c1bc3cc515493993daffbdbf15ee20'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:48.888306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:48.902840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 899/1809] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:48 2022] DELETE /identity/v3/users/b8c1bc3cc515493993daffbdbf15ee20 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:50.105063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:50.107617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:50.121528 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82d36-174d-43e1-8203-a944e1f35e88 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:50.200554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:50.212659 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:50.213352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.214003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.215185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] PATH_INFO: `/v3/projects/cd2693b5ab5c451798cac69ca7a2e679` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.218525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2693b5ab5c451798cac69ca7a2e679)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:50.230200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:50.325574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2693b5ab5c451798cac69ca7a2e679', 'request_id': 'req-5c0e30ab-146b-46cd-a8d0-53c361d4a896'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:50.333779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2693b5ab5c451798cac69ca7a2e679', 'request_id': 'req-5c0e30ab-146b-46cd-a8d0-53c361d4a896'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:50.342003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2693b5ab5c451798cac69ca7a2e679', 'request_id': 'req-5c0e30ab-146b-46cd-a8d0-53c361d4a896'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:50.364414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] The token cache is being invalidate because project cd2693b5ab5c451798cac69ca7a2e679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:50.364823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:50.370158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c0e30ab-146b-46cd-a8d0-53c361d4a896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:50.372557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 911/1810] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:50 2022] DELETE /identity/v3/projects/cd2693b5ab5c451798cac69ca7a2e679 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:50.378440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d828224-2fae-495b-a593-49ff56372586 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.379276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d828224-2fae-495b-a593-49ff56372586 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.379748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d828224-2fae-495b-a593-49ff56372586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.403826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9d828224-2fae-495b-a593-49ff56372586 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:50.409602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9d828224-2fae-495b-a593-49ff56372586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:50.490140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 900/1811] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:50.497747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f244fbf-38e7-45bd-87c8-782c8b6e73e7 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.498089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f244fbf-38e7-45bd-87c8-782c8b6e73e7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.498816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f244fbf-38e7-45bd-87c8-782c8b6e73e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.519909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3f244fbf-38e7-45bd-87c8-782c8b6e73e7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:50.521405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3f244fbf-38e7-45bd-87c8-782c8b6e73e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:50.532778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 912/1812] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:50.538183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d828224-2fae-495b-a593-49ff56372586 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:50.558915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d828224-2fae-495b-a593-49ff56372586 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:50.567774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:50.569120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.570157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.571399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.572331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:50.572732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:50.574649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:50.598348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 901/1813] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:38:50.601423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f244fbf-38e7-45bd-87c8-782c8b6e73e7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:50.614463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f244fbf-38e7-45bd-87c8-782c8b6e73e7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:50.626141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ea537-c7cc-49d2-89c9-280f106627c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:50.626872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ea537-c7cc-49d2-89c9-280f106627c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.627056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ea537-c7cc-49d2-89c9-280f106627c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.627443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ea537-c7cc-49d2-89c9-280f106627c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.628153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ea537-c7cc-49d2-89c9-280f106627c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:50.631776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ea537-c7cc-49d2-89c9-280f106627c5 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:50.708744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 913/1814] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:50 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:50.713197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:50.731095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd4745-fbdc-4ce0-b634-2ca84c33f3ec admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:50.804023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6d0d-7ae7-46de-94ec-154620c4fab3 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.804313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6d0d-7ae7-46de-94ec-154620c4fab3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.805294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6d0d-7ae7-46de-94ec-154620c4fab3 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.807291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 914/1815] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:38:50.824617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411712c-6b21-40e5-b2d6-d91ffb355d9e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.824938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411712c-6b21-40e5-b2d6-d91ffb355d9e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.825221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411712c-6b21-40e5-b2d6-d91ffb355d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.837798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:50.840487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.841608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.844047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.845925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:50.850651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:50.863913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3411712c-6b21-40e5-b2d6-d91ffb355d9e None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:50.865426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3411712c-6b21-40e5-b2d6-d91ffb355d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:50.911925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 902/1816] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:50 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:50.937370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:50.965072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faf6f0-1162-40af-bd6d-ada3c29e191e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:50.984215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:50.984763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:50.985677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:50.985955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:50.986740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:50.988581 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.021093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 903/1817] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:51.022063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.074529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 915/1818] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:38:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:38:51.075735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03a027-f9ff-4c8a-a0d3-6b5a2977d13b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:51.092070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e9a0-0d65-4fee-8175-d87b90832126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:51.093920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e9a0-0d65-4fee-8175-d87b90832126 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.094038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e9a0-0d65-4fee-8175-d87b90832126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.094269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e9a0-0d65-4fee-8175-d87b90832126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.096372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e9a0-0d65-4fee-8175-d87b90832126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:51.099519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e9a0-0d65-4fee-8175-d87b90832126 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.126703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 904/1819] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:51.130847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411712c-6b21-40e5-b2d6-d91ffb355d9e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.148843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411712c-6b21-40e5-b2d6-d91ffb355d9e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:51.177247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:51.178637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.178923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.180430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] PATH_INFO: `/v3/projects/219a729f589a4b0799639fb962a5f3b8/users/cf6998df9d6c47fd9a2e136539dd0cfb/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.216427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=219a729f589a4b0799639fb962a5f3b8, user_id=cf6998df9d6c47fd9a2e136539dd0cfb, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:51.223158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.250385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 916/1820] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:51 2022] PUT /identity/v3/projects/219a729f589a4b0799639fb962a5f3b8/users/cf6998df9d6c47fd9a2e136539dd0cfb/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:51.261716 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24115b02-df7c-4564-b28b-6444bd9f1417 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.262070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24115b02-df7c-4564-b28b-6444bd9f1417 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.262416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24115b02-df7c-4564-b28b-6444bd9f1417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.347390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.348533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-24115b02-df7c-4564-b28b-6444bd9f1417 None None] MFA Rules not processed for user `cf6998df9d6c47fd9a2e136539dd0cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:51.349765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-24115b02-df7c-4564-b28b-6444bd9f1417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:51.351877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:51.401925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:51.455520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:51.489060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b18561-c516-49f3-8f98-801dcda99c68 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:51.514372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:51.515218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.515510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.515829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.522779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 905/1821] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:51.526146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24115b02-df7c-4564-b28b-6444bd9f1417 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.526935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:51.529172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.553453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 917/1822] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:38:51.646846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24115b02-df7c-4564-b28b-6444bd9f1417 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:51.652821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:51.653027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.653208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.653875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.654336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:51.656413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.707311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 906/1823] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:51 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:51.711223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.722685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01217fa9-c444-49d5-8f18-ec2f2cf2a7a8 service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:51.789123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:51.789948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.790094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.790495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.791389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:51.793373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.829421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 918/1824] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:51 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:51.830370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.842397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6da2-68c9-4b84-bd31-0454135e7811 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:51.847672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:51.848374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.848674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.849031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.849811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:51.851640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.863722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 907/1825] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:51.869282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.882235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2f24c-fd45-4b08-b1cd-c7d0e887ecaf admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:51.890391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:51.891046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:51.891357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:51.891667 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] PATH_INFO: `/v3/projects/e8594c3d31dc40d1be355aa03be59359/users/620ca53e91a84038891e6c2fcd7ffe10/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:51.913571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8594c3d31dc40d1be355aa03be59359, user_id=620ca53e91a84038891e6c2fcd7ffe10, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:51.917174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:51.939973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 919/1826] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:51 2022] PUT /identity/v3/projects/e8594c3d31dc40d1be355aa03be59359/users/620ca53e91a84038891e6c2fcd7ffe10/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:51.940973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:51.957283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61af7dc-cff5-4013-b16b-d17c3ba046bd admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:52.020978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f1180-ec18-45f0-b764-73134cb60d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:52.021599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f1180-ec18-45f0-b764-73134cb60d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:52.021872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f1180-ec18-45f0-b764-73134cb60d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:52.022149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f1180-ec18-45f0-b764-73134cb60d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:52.022799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f1180-ec18-45f0-b764-73134cb60d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:52.024435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f1180-ec18-45f0-b764-73134cb60d8b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:52.034478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 908/1827] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:38:52.038775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:52.049885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292f9c-b7ad-439c-bfd6-3dc125b1d1cd admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:52.054475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:52.055132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:52.055413 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:52.055728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] PATH_INFO: `/v3/projects/e8594c3d31dc40d1be355aa03be59359/users/620ca53e91a84038891e6c2fcd7ffe10/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:52.057826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8594c3d31dc40d1be355aa03be59359, user_id=620ca53e91a84038891e6c2fcd7ffe10, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:52.060503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:52.071603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 920/1828] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:52 2022] PUT /identity/v3/projects/e8594c3d31dc40d1be355aa03be59359/users/620ca53e91a84038891e6c2fcd7ffe10/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:52.083857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5a87f-84b4-48df-ad53-90622810b601 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:52.084142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5a87f-84b4-48df-ad53-90622810b601 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:52.084411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5a87f-84b4-48df-ad53-90622810b601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:52.113827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-63b5a87f-84b4-48df-ad53-90622810b601 None None] MFA Rules not processed for user `620ca53e91a84038891e6c2fcd7ffe10`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:52.115057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-63b5a87f-84b4-48df-ad53-90622810b601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:52.157268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:52.188808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 909/1829] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:52.201763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:52.203944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:52.263877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f14a8-c8be-4bdf-8efc-bf602d3c8ddd admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:52.267800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:52.268409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:52.268712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:52.268976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:52.278959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:52.280514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:52.293757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 921/1830] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:52.750508 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5a87f-84b4-48df-ad53-90622810b601 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:52.764333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5a87f-84b4-48df-ad53-90622810b601 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:52.770038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620ca53e91a84038891e6c2fcd7ffe10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8594c3d31dc40d1be355aa03be59359', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:52.770708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:52.771833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:52.771960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:52.772991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:52.773410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:52.775273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:52.797199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:52.803436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 910/1831] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:52.803992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:52.809344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708154c-c6d2-45b6-8421-1cd0ace086d8 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:52.816965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620ca53e91a84038891e6c2fcd7ffe10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8594c3d31dc40d1be355aa03be59359', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:52.817767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:52.818007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:52.818281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:52.818915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:52.820866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:52.855903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 922/1832] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:52 2022] POST /identity/v3/projects => generated 351 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:52.857865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:52.869219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1a245-ae5d-4945-9c62-8afaead5b062 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:52.954335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620ca53e91a84038891e6c2fcd7ffe10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8594c3d31dc40d1be355aa03be59359', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:52.955040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:52.955559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:52.955736 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:52.956468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:52.956839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:52.958670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:52.972738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 911/1833] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:53.021559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:53.033180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622b46c-011a-42a7-9e4e-0d4155d2dff4 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:53.037851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '620ca53e91a84038891e6c2fcd7ffe10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8594c3d31dc40d1be355aa03be59359', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:53.038531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:53.038816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:53.039093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] PATH_INFO: `/v3/projects/913f52131097415e928288c422b6dcc8` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:53.040278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorizing `identity:delete_project(project_id=913f52131097415e928288c422b6dcc8)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:53.042283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:53.095449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913f52131097415e928288c422b6dcc8', 'request_id': 'req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.101171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913f52131097415e928288c422b6dcc8', 'request_id': 'req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.107765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913f52131097415e928288c422b6dcc8', 'request_id': 'req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.122283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] The token cache is being invalidate because project 913f52131097415e928288c422b6dcc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:53.122565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.123411 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9bcfee32-ef4d-4d9a-a602-b0cf3a5c6cc0 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.125158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 923/1834] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:53 2022] DELETE /identity/v3/projects/913f52131097415e928288c422b6dcc8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:53.341244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:53.343903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:53.413665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb1b1a-ef7f-4ae5-a1c6-36b8e9b879a9 tempest-VolumeQuotaClassesTest-1928657282 tempest-VolumeQuotaClassesTest-1928657282-project-admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:53.474326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:53.474976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:53.475276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:53.475543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] PATH_INFO: `/v3/users/cf6998df9d6c47fd9a2e136539dd0cfb` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:53.476713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6998df9d6c47fd9a2e136539dd0cfb)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:53.478668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:53.483670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:53.529156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6998df9d6c47fd9a2e136539dd0cfb'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.529926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6998df9d6c47fd9a2e136539dd0cfb'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.530626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6998df9d6c47fd9a2e136539dd0cfb'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.539787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6998df9d6c47fd9a2e136539dd0cfb'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.547323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6998df9d6c47fd9a2e136539dd0cfb'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:53.564482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 912/1835] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:53 2022] DELETE /identity/v3/users/cf6998df9d6c47fd9a2e136539dd0cfb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:53.569069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:53.569532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:53.569643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:53.588939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:53.590123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:53.654592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 924/1836] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:53.665100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:53.666623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:53.730487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9b4cb-895d-4048-bb6d-375e3bb08b02 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:53.735144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:53.740093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:53.740380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:53.740670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:53.750250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:53.751686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:53.760977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 913/1837] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:38:54.190666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:54.192615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:54.216640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2fdbd-987f-4b39-9223-2a965416119e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:54.222210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:54.222858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:54.223140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:54.223410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] PATH_INFO: `/v3/projects/219a729f589a4b0799639fb962a5f3b8` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:54.224569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] RBAC: Authorizing `identity:delete_project(project_id=219a729f589a4b0799639fb962a5f3b8)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:54.226471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:54.279872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '219a729f589a4b0799639fb962a5f3b8', 'request_id': 'req-628bd9fa-b6cb-4217-b987-34e2a910795b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.286142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '219a729f589a4b0799639fb962a5f3b8', 'request_id': 'req-628bd9fa-b6cb-4217-b987-34e2a910795b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.288469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:54.293319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '219a729f589a4b0799639fb962a5f3b8', 'request_id': 'req-628bd9fa-b6cb-4217-b987-34e2a910795b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.306922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] The token cache is being invalidate because project 219a729f589a4b0799639fb962a5f3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:54.307205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.307846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.309353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 925/1838] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:54 2022] DELETE /identity/v3/projects/219a729f589a4b0799639fb962a5f3b8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:54.314108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:54.323598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c46e05-1148-4019-ba6f-d06f137d6323 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:54.382683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:54.383317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:54.383609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:54.383892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] PATH_INFO: `/v3/users/620ca53e91a84038891e6c2fcd7ffe10` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:54.385001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] RBAC: Authorizing `identity:delete_user(user_id=620ca53e91a84038891e6c2fcd7ffe10)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:54.386936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:54.419206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:54.438992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620ca53e91a84038891e6c2fcd7ffe10'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.439671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620ca53e91a84038891e6c2fcd7ffe10'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.440277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620ca53e91a84038891e6c2fcd7ffe10'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.448462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620ca53e91a84038891e6c2fcd7ffe10'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.455449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f3ea35ce-1999-47fa-9048-0a8ec659622d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620ca53e91a84038891e6c2fcd7ffe10'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.473759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 914/1839] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:54 2022] DELETE /identity/v3/users/620ca53e91a84038891e6c2fcd7ffe10 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:54.764305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:54.770984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:54.785214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd9fa-b6cb-4217-b987-34e2a910795b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:54.851712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:54.852658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:54.853027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:54.853404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] PATH_INFO: `/v3/projects/e8594c3d31dc40d1be355aa03be59359` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:54.855338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8594c3d31dc40d1be355aa03be59359)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:54.858705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:54.912127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8594c3d31dc40d1be355aa03be59359', 'request_id': 'req-5e895308-2ef3-426e-a200-faeb99f5b98b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.918660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8594c3d31dc40d1be355aa03be59359', 'request_id': 'req-5e895308-2ef3-426e-a200-faeb99f5b98b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.925609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8594c3d31dc40d1be355aa03be59359', 'request_id': 'req-5e895308-2ef3-426e-a200-faeb99f5b98b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.940911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] The token cache is being invalidate because project e8594c3d31dc40d1be355aa03be59359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:54.941464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.942088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5e895308-2ef3-426e-a200-faeb99f5b98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:54.943610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 926/1840] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:54 2022] DELETE /identity/v3/projects/e8594c3d31dc40d1be355aa03be59359 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:54.948654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c1888-3206-4ce6-b764-6220e07fd3d6 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:54.949141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c1888-3206-4ce6-b764-6220e07fd3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:54.949240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c1888-3206-4ce6-b764-6220e07fd3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:54.970178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-b54c1888-3206-4ce6-b764-6220e07fd3d6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:54.971391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b54c1888-3206-4ce6-b764-6220e07fd3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:55.034499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 915/1841] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:55.044570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19b20-ebae-4d43-8d67-902470954525 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.044859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19b20-ebae-4d43-8d67-902470954525 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.045186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19b20-ebae-4d43-8d67-902470954525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.065773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c1d19b20-ebae-4d43-8d67-902470954525 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:55.066973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c1d19b20-ebae-4d43-8d67-902470954525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:55.076329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 927/1842] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:55.081537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c1888-3206-4ce6-b764-6220e07fd3d6 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.096326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c1888-3206-4ce6-b764-6220e07fd3d6 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.102263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.102934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.103150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.112157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.112840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:55.113168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.114815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.128472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 916/1843] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:55.140409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d19b20-ebae-4d43-8d67-902470954525 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.153548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d19b20-ebae-4d43-8d67-902470954525 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.158656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.159277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.159550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.159862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.160489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.162660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.230446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 928/1844] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:55 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:55.234343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.248986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f765092-fc22-48af-9ba3-44003ebe6897 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.309856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.310519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.310791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.311502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.312362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.314278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.338983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 917/1845] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:55 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:55.342969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.354522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482643ce-fd86-4ffc-b7b2-1057c38e53b1 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.359533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.360241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.360526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.360853 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.361526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.363311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.384915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 929/1846] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:55.386216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.400946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94665-c4f5-4001-931d-3bb04956c0b9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.405921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851dc6f5-59d0-400b-92d6-a40cfcab2d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.406557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dc6f5-59d0-400b-92d6-a40cfcab2d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.406823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dc6f5-59d0-400b-92d6-a40cfcab2d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.407088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dc6f5-59d0-400b-92d6-a40cfcab2d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.407770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851dc6f5-59d0-400b-92d6-a40cfcab2d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.409374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851dc6f5-59d0-400b-92d6-a40cfcab2d42 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.420770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 918/1847] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:55.425212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.435119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f44dc-61f8-4bc9-b6c4-85dd6a4422d1 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.440024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.440701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.440960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.441220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] PATH_INFO: `/v3/projects/07f2c0325232478a9ede19e35c340974/users/fa4b808737d94d329faa8ace0404fd94/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.456766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f2c0325232478a9ede19e35c340974, user_id=fa4b808737d94d329faa8ace0404fd94, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.459405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.474079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 930/1848] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:55 2022] PUT /identity/v3/projects/07f2c0325232478a9ede19e35c340974/users/fa4b808737d94d329faa8ace0404fd94/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:55.484702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8d5c-b91a-410d-bbdd-02f7909d5f38 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.484972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8d5c-b91a-410d-bbdd-02f7909d5f38 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.485263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8d5c-b91a-410d-bbdd-02f7909d5f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.507645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:55.516131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-50fd8d5c-b91a-410d-bbdd-02f7909d5f38 None None] MFA Rules not processed for user `fa4b808737d94d329faa8ace0404fd94`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:55.518106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-50fd8d5c-b91a-410d-bbdd-02f7909d5f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:55.583991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 919/1849] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:55.588636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.611555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:55.657450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b32a1-9f4e-47b0-8f5f-85a985f44c99 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.662168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.662780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.663057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.663324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.663978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.665893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.703149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 931/1850] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:55 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:55.706715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8d5c-b91a-410d-bbdd-02f7909d5f38 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.718762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8d5c-b91a-410d-bbdd-02f7909d5f38 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.783285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.783972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.784253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.784528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.785167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.787185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.811557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 920/1851] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:55 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:55.815811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.826663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1b28c-2848-4d7f-9238-b51d3e6e71dd admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.831884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.832527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.832800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.833064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.833711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.835363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.847190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 932/1852] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:55.853380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.864811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc082d9e-b48f-4358-b5ec-6e29ed783b63 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.870978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.871727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.872011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.872269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] PATH_INFO: `/v3/projects/c7af5abc0c2a4c7fad2709f894ffad57/users/9784b270c6d647d58ecc3afd80323a36/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.889577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7af5abc0c2a4c7fad2709f894ffad57, user_id=9784b270c6d647d58ecc3afd80323a36, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.892490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:55.908253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 921/1853] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:55 2022] PUT /identity/v3/projects/c7af5abc0c2a4c7fad2709f894ffad57/users/9784b270c6d647d58ecc3afd80323a36/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:55.912403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:55.922913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60fd6c-e9a7-45ff-a963-4910fb04055d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:55.992433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944be5-65bf-47f6-80a6-57c495843842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:55.993055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944be5-65bf-47f6-80a6-57c495843842 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:55.993305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944be5-65bf-47f6-80a6-57c495843842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:55.993549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944be5-65bf-47f6-80a6-57c495843842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:55.994185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944be5-65bf-47f6-80a6-57c495843842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:55.995891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944be5-65bf-47f6-80a6-57c495843842 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:56.006253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 933/1854] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:56.011432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:56.022278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ae60-ef09-47bd-a22c-0a204980793f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:56.028046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53defe78-9999-4914-beef-2622b488391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:56.028741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53defe78-9999-4914-beef-2622b488391c admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:56.029021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53defe78-9999-4914-beef-2622b488391c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:56.029295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53defe78-9999-4914-beef-2622b488391c admin admin] PATH_INFO: `/v3/projects/c7af5abc0c2a4c7fad2709f894ffad57/users/9784b270c6d647d58ecc3afd80323a36/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:56.031862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53defe78-9999-4914-beef-2622b488391c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7af5abc0c2a4c7fad2709f894ffad57, user_id=9784b270c6d647d58ecc3afd80323a36, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:56.034474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53defe78-9999-4914-beef-2622b488391c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:56.045059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 922/1855] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:56 2022] PUT /identity/v3/projects/c7af5abc0c2a4c7fad2709f894ffad57/users/9784b270c6d647d58ecc3afd80323a36/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:56.056242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d532edb-0f08-4342-b37b-b9febf49249f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:56.056638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d532edb-0f08-4342-b37b-b9febf49249f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:56.056914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d532edb-0f08-4342-b37b-b9febf49249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:56.091926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3d532edb-0f08-4342-b37b-b9febf49249f None None] MFA Rules not processed for user `9784b270c6d647d58ecc3afd80323a36`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:56.093999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3d532edb-0f08-4342-b37b-b9febf49249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:56.167228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 934/1856] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:56.179102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53defe78-9999-4914-beef-2622b488391c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:56.181018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-53defe78-9999-4914-beef-2622b488391c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:56.253045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53defe78-9999-4914-beef-2622b488391c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:56.256753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564663af-f25e-40ea-a0a4-534ebc2f7f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:56.257375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564663af-f25e-40ea-a0a4-534ebc2f7f5f service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:56.257655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564663af-f25e-40ea-a0a4-534ebc2f7f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:56.257926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564663af-f25e-40ea-a0a4-534ebc2f7f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:56.271148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:56.271148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564663af-f25e-40ea-a0a4-534ebc2f7f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:56.271148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564663af-f25e-40ea-a0a4-534ebc2f7f5f service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:56.280859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 923/1857] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:56.905166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d532edb-0f08-4342-b37b-b9febf49249f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:56.918624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:56.969014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d532edb-0f08-4342-b37b-b9febf49249f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:56.974660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:56.975303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:56.975603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:56.975878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] PATH_INFO: `/v3/users/fa4b808737d94d329faa8ace0404fd94` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:56.977804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4b808737d94d329faa8ace0404fd94)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:56.979873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:57.030781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4b808737d94d329faa8ace0404fd94'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.031440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4b808737d94d329faa8ace0404fd94'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.038090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4b808737d94d329faa8ace0404fd94'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.038699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4b808737d94d329faa8ace0404fd94'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.048893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4b808737d94d329faa8ace0404fd94'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.065539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 935/1858] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:56 2022] DELETE /identity/v3/users/fa4b808737d94d329faa8ace0404fd94 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:57.071481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:57.072007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:57.072281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:57.092813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:57.094498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:57.156614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 924/1859] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:57.166814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:57.168632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:57.227507 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dfb5e-9bbc-4427-af83-ab148d6de5bf admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:57.231735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:57.232355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:57.232640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:57.232918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:57.242580 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:57.244061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:57.254093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 936/1860] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:38:57.548044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:57.549515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:57.565241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff17dc-f03f-47fa-9dee-64a711b2e2dc None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:57.571734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:57.572482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:57.572682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:57.572918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] PATH_INFO: `/v3/projects/07f2c0325232478a9ede19e35c340974` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:57.574192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f2c0325232478a9ede19e35c340974)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:57.575996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:57.628595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f2c0325232478a9ede19e35c340974', 'request_id': 'req-b59b6a4a-2724-4411-be9e-52414e136c13'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.634317 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f2c0325232478a9ede19e35c340974', 'request_id': 'req-b59b6a4a-2724-4411-be9e-52414e136c13'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.640978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f2c0325232478a9ede19e35c340974', 'request_id': 'req-b59b6a4a-2724-4411-be9e-52414e136c13'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.654322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] The token cache is being invalidate because project 07f2c0325232478a9ede19e35c340974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:57.654636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.655313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.657536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 925/1861] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:57 2022] DELETE /identity/v3/projects/07f2c0325232478a9ede19e35c340974 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:57.661295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:57.676605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15739ae9-b25f-4c4d-94f5-1ec9142f1744 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:57.710923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:57.740589 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:57.741139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:57.741414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:57.741682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] PATH_INFO: `/v3/users/9784b270c6d647d58ecc3afd80323a36` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:57.742789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9784b270c6d647d58ecc3afd80323a36)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:57.744906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:57.800581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9784b270c6d647d58ecc3afd80323a36'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.801260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9784b270c6d647d58ecc3afd80323a36'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.808658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9784b270c6d647d58ecc3afd80323a36'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.809489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9784b270c6d647d58ecc3afd80323a36'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.820120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ace42fae-6668-412f-a486-03136043cd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9784b270c6d647d58ecc3afd80323a36'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:57.856309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 937/1862] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:38:57 2022] DELETE /identity/v3/users/9784b270c6d647d58ecc3afd80323a36 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:58.222107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.223736 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:58.236594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b6a4a-2724-4411-be9e-52414e136c13 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:58.299325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:58.299966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.300243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.300487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] PATH_INFO: `/v3/projects/c7af5abc0c2a4c7fad2709f894ffad57` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.301713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7af5abc0c2a4c7fad2709f894ffad57)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:58.303674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:58.329092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:58.358877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7af5abc0c2a4c7fad2709f894ffad57', 'request_id': 'req-14a37491-7237-4f79-8bc8-c5168ce742f0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:58.365326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7af5abc0c2a4c7fad2709f894ffad57', 'request_id': 'req-14a37491-7237-4f79-8bc8-c5168ce742f0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:58.373691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7af5abc0c2a4c7fad2709f894ffad57', 'request_id': 'req-14a37491-7237-4f79-8bc8-c5168ce742f0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:58.388915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] The token cache is being invalidate because project c7af5abc0c2a4c7fad2709f894ffad57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:38:58.389208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:58.389910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-14a37491-7237-4f79-8bc8-c5168ce742f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:38:58.391624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 926/1863] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:38:58 2022] DELETE /identity/v3/projects/c7af5abc0c2a4c7fad2709f894ffad57 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:38:58.397477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09917-f7ac-435d-aede-a7c3951446ec None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.397945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09917-f7ac-435d-aede-a7c3951446ec None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.398134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09917-f7ac-435d-aede-a7c3951446ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.417901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c2c09917-f7ac-435d-aede-a7c3951446ec None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:58.419242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c2c09917-f7ac-435d-aede-a7c3951446ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:58.483090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 938/1864] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:58.492475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ce1c-3767-4293-baca-a7cf6054423c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.492758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ce1c-3767-4293-baca-a7cf6054423c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.493049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ce1c-3767-4293-baca-a7cf6054423c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.514238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f955ce1c-3767-4293-baca-a7cf6054423c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:58.515500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f955ce1c-3767-4293-baca-a7cf6054423c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:58.523530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 927/1865] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:58.527765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c09917-f7ac-435d-aede-a7c3951446ec None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.537549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c09917-f7ac-435d-aede-a7c3951446ec None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:58.542193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:58.542788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.543070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.543343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.544028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:38:58.544328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:58.545944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:58.560627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 939/1866] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:38:58.566991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955ce1c-3767-4293-baca-a7cf6054423c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.576496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955ce1c-3767-4293-baca-a7cf6054423c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:58.582399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:58.583003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.583280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.583543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.584213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:58.586100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:58.615985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 928/1867] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:38:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:58.620247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.630859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3604c6e-120d-4cef-81d6-d6714c7a34cc admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:58.675271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:38:58.694966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:58.695630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.695894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.696156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.696768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:58.698687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:58.722422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 940/1868] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:38:58 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:38:58.726679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.737184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d6a1c-ae1d-4a55-871b-96ad144e5c12 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:58.741874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:58.742471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.742746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.743014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.743714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:58.745300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:58.755765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 929/1869] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:38:58.760083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.771729 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a4446-2710-40f2-901c-e7bf28061d7a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:58.776973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0227d-53a8-43f7-8992-f999e9314b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:58.777567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0227d-53a8-43f7-8992-f999e9314b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.777826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0227d-53a8-43f7-8992-f999e9314b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.778124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0227d-53a8-43f7-8992-f999e9314b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.778806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0227d-53a8-43f7-8992-f999e9314b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:58.780425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0227d-53a8-43f7-8992-f999e9314b11 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:58.790289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 941/1870] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:38:58.794420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.804666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669c58-c6a8-4f0a-b560-b358db449785 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:58.810585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:58.811421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.811647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.812307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] PATH_INFO: `/v3/projects/1da62c2f260d436b9151764cbe5ce1e4/users/629d2a026c0f4f2b99acc345bec91bc8/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.830121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da62c2f260d436b9151764cbe5ce1e4, user_id=629d2a026c0f4f2b99acc345bec91bc8, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:58.832834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:58.852647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 930/1871] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:38:58 2022] PUT /identity/v3/projects/1da62c2f260d436b9151764cbe5ce1e4/users/629d2a026c0f4f2b99acc345bec91bc8/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:38:58.863472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2693dd-500e-4c6e-9548-b8f5d21c2520 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:58.863791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2693dd-500e-4c6e-9548-b8f5d21c2520 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:58.864063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2693dd-500e-4c6e-9548-b8f5d21c2520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:58.896788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7a2693dd-500e-4c6e-9548-b8f5d21c2520 None None] MFA Rules not processed for user `629d2a026c0f4f2b99acc345bec91bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:38:58.898055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7a2693dd-500e-4c6e-9548-b8f5d21c2520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:58.973337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 942/1872] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:38:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:38:58.985228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:38:58.986915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:38:59.049506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ee09c-793b-49f8-83f6-33f3e74345ab admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:38:59.054348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c4162-711c-43a2-b837-8a35372520ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:38:59.055163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c4162-711c-43a2-b837-8a35372520ed service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:38:59.060722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c4162-711c-43a2-b837-8a35372520ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:38:59.061015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c4162-711c-43a2-b837-8a35372520ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:38:59.071805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c4162-711c-43a2-b837-8a35372520ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:38:59.073442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c4162-711c-43a2-b837-8a35372520ed service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:38:59.083396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 931/1873] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:38:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:39:05.410077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90e160-8910-431a-8b7b-36b86b969db3 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:05.411081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90e160-8910-431a-8b7b-36b86b969db3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:05.411613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90e160-8910-431a-8b7b-36b86b969db3 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:05.414874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 943/1874] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:39:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:39:05.424444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752488-814d-433a-b71e-a87883efcf80 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:05.425602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752488-814d-433a-b71e-a87883efcf80 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:05.425882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752488-814d-433a-b71e-a87883efcf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:05.452354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-79752488-814d-433a-b71e-a87883efcf80 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:05.453846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-79752488-814d-433a-b71e-a87883efcf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:05.529197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:05.556935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 932/1875] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:39:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:05.672310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90e160-8910-431a-8b7b-36b86b969db3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:05.672310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-af90e160-8910-431a-8b7b-36b86b969db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:05.688006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90e160-8910-431a-8b7b-36b86b969db3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:05.693899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fda9d-eecf-462e-81db-9356351448c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:05.694545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fda9d-eecf-462e-81db-9356351448c0 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:05.694822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fda9d-eecf-462e-81db-9356351448c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:05.695094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fda9d-eecf-462e-81db-9356351448c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:05.704466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fda9d-eecf-462e-81db-9356351448c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:05.706057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fda9d-eecf-462e-81db-9356351448c0 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:05.716991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 944/1876] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:09.669011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752488-814d-433a-b71e-a87883efcf80 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:09.808261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752488-814d-433a-b71e-a87883efcf80 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:09.816207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:09.828094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:09.828546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:09.829179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] PATH_INFO: `/v3/users/629d2a026c0f4f2b99acc345bec91bc8` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:09.833897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=629d2a026c0f4f2b99acc345bec91bc8)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:09.836423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:09.938575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629d2a026c0f4f2b99acc345bec91bc8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:09.939565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629d2a026c0f4f2b99acc345bec91bc8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:09.941667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629d2a026c0f4f2b99acc345bec91bc8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:09.972354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629d2a026c0f4f2b99acc345bec91bc8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:09.983824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629d2a026c0f4f2b99acc345bec91bc8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:10.023908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 933/1877] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:09 2022] DELETE /identity/v3/users/629d2a026c0f4f2b99acc345bec91bc8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:10.026450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16825539-4b50-4482-bedc-fc651721e550 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:10.027084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16825539-4b50-4482-bedc-fc651721e550 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:10.027534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16825539-4b50-4482-bedc-fc651721e550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:10.063325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-16825539-4b50-4482-bedc-fc651721e550 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:10.065077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-16825539-4b50-4482-bedc-fc651721e550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:10.146536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 945/1878] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:10.155533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:10.157529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:10.225925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cb670-0359-4935-bd9b-d510492fa4f3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:10.229711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86157b-2262-40c7-87e1-16bf56741c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:10.230559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86157b-2262-40c7-87e1-16bf56741c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:10.230931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86157b-2262-40c7-87e1-16bf56741c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:10.231265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86157b-2262-40c7-87e1-16bf56741c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:10.243706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86157b-2262-40c7-87e1-16bf56741c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:10.245442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86157b-2262-40c7-87e1-16bf56741c2a service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:10.256359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 934/1879] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:39:10.538295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16825539-4b50-4482-bedc-fc651721e550 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:10.539921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-16825539-4b50-4482-bedc-fc651721e550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:10.554830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16825539-4b50-4482-bedc-fc651721e550 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:10.560123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:10.560770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:10.561091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:10.561310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] PATH_INFO: `/v3/projects/1da62c2f260d436b9151764cbe5ce1e4` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:10.562481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da62c2f260d436b9151764cbe5ce1e4)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:10.564433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:10.607140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:10.625153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da62c2f260d436b9151764cbe5ce1e4', 'request_id': 'req-f97aa260-c992-40bb-be16-4fb448cabe01'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:10.635452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da62c2f260d436b9151764cbe5ce1e4', 'request_id': 'req-f97aa260-c992-40bb-be16-4fb448cabe01'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:10.642644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da62c2f260d436b9151764cbe5ce1e4', 'request_id': 'req-f97aa260-c992-40bb-be16-4fb448cabe01'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:10.657137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] The token cache is being invalidate because project 1da62c2f260d436b9151764cbe5ce1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:10.657759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:10.660490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f97aa260-c992-40bb-be16-4fb448cabe01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:10.663035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 946/1880] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:10 2022] DELETE /identity/v3/projects/1da62c2f260d436b9151764cbe5ce1e4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:10.671869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fb40-3153-43ef-aa9d-e1daecfb58bd None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:10.671869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fb40-3153-43ef-aa9d-e1daecfb58bd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:10.671869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fb40-3153-43ef-aa9d-e1daecfb58bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:10.679762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:10.701595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-37b4fb40-3153-43ef-aa9d-e1daecfb58bd None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:10.702853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-37b4fb40-3153-43ef-aa9d-e1daecfb58bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:10.885490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 935/1881] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:10.895307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9064e-b2c7-4108-b631-dc8728d7145e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:10.895746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9064e-b2c7-4108-b631-dc8728d7145e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:10.899750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9064e-b2c7-4108-b631-dc8728d7145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:10.921157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d6a9064e-b2c7-4108-b631-dc8728d7145e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:10.922403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d6a9064e-b2c7-4108-b631-dc8728d7145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:10.931728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 947/1882] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:10.935810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fb40-3153-43ef-aa9d-e1daecfb58bd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:10.946740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fb40-3153-43ef-aa9d-e1daecfb58bd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:10.964993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:10.965637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:10.966046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:10.966348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:10.967011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:10.967562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:10.969456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:10.982916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 936/1883] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:39:10.988082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9064e-b2c7-4108-b631-dc8728d7145e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.003629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9064e-b2c7-4108-b631-dc8728d7145e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.008659 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.010182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.010894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.011091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.012074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.014460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.066070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 948/1884] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:10 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:11.068214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.079515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c53d5-8672-4e53-b852-816f8a740d3e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.156970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.157710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.158048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.158380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.159073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.161181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.185160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 937/1885] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:11 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:11.188601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.198664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00e60e-cf31-44bd-895b-2b2e2dcb9592 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.203357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.204708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.205003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.205278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.205935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.207543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.217672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 949/1886] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:11.222182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.232022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba2d09-5403-432a-a203-742e44441e5b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.237408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af409f-25c3-4399-8d97-70aabf6e1d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.238010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af409f-25c3-4399-8d97-70aabf6e1d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.238289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af409f-25c3-4399-8d97-70aabf6e1d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.238559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af409f-25c3-4399-8d97-70aabf6e1d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.239201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af409f-25c3-4399-8d97-70aabf6e1d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.240840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af409f-25c3-4399-8d97-70aabf6e1d5a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.252241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 938/1887] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:11.256260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.266702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0f97-f105-4adf-97f8-b2d80c37081d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.272452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.273109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.273384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.273654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] PATH_INFO: `/v3/projects/23b4e95b667e46e9a1556ec7a6d8b113/users/2d5f18eb43214ab68482ba8b0f7deeb5/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.291426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b4e95b667e46e9a1556ec7a6d8b113, user_id=2d5f18eb43214ab68482ba8b0f7deeb5, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.294172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.313135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 950/1888] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:11 2022] PUT /identity/v3/projects/23b4e95b667e46e9a1556ec7a6d8b113/users/2d5f18eb43214ab68482ba8b0f7deeb5/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:11.328133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f5712-4a31-4c55-8f97-7f554d5413c1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.328590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f5712-4a31-4c55-8f97-7f554d5413c1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.328860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f5712-4a31-4c55-8f97-7f554d5413c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.361717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5d2f5712-4a31-4c55-8f97-7f554d5413c1 None None] MFA Rules not processed for user `2d5f18eb43214ab68482ba8b0f7deeb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:11.362910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5d2f5712-4a31-4c55-8f97-7f554d5413c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:11.432601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:11.438461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 939/1889] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:11.444590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.516802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7414fcd-7187-4628-8b0e-8ac6c67309e7 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.522409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.523107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.523395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.523731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.524355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.526290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.559499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 951/1890] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:11 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:11.564056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f5712-4a31-4c55-8f97-7f554d5413c1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.574901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f5712-4a31-4c55-8f97-7f554d5413c1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.631797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.632407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.632666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.632937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.633575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.635489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.658864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 940/1891] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:11 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:11.663115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.673543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228256d3-de25-472d-be15-e1fbe1574bd5 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.678127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.678733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.679009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.679281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.680013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.681628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.692368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 952/1892] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:11.696349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.706914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15695c0e-4215-4b46-8254-14fc9bbb5778 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.712711 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba3195-c35d-4809-8551-040bece10fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.713307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba3195-c35d-4809-8551-040bece10fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.713589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba3195-c35d-4809-8551-040bece10fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.713858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba3195-c35d-4809-8551-040bece10fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.714513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba3195-c35d-4809-8551-040bece10fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.716115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba3195-c35d-4809-8551-040bece10fb0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.729035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 941/1893] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:11.733781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.748682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c3d0e-e4a7-4f4c-a0a4-71134aa3d794 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.754177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.754824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.755100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.755374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] PATH_INFO: `/v3/projects/401bfe45dca84dbab522f6a25d8daf96/users/5ff89cd266ba4173936412aa08dee3f3/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.770336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=401bfe45dca84dbab522f6a25d8daf96, user_id=5ff89cd266ba4173936412aa08dee3f3, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.773103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:11.788113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 953/1894] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:11 2022] PUT /identity/v3/projects/401bfe45dca84dbab522f6a25d8daf96/users/5ff89cd266ba4173936412aa08dee3f3/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:11.799054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af8b1-2366-4c21-8b13-947961e86b55 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.799344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af8b1-2366-4c21-8b13-947961e86b55 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.799617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af8b1-2366-4c21-8b13-947961e86b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.832208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-359af8b1-2366-4c21-8b13-947961e86b55 None None] MFA Rules not processed for user `5ff89cd266ba4173936412aa08dee3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:11.833423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-359af8b1-2366-4c21-8b13-947961e86b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:11.905389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 942/1895] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:11.910267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:11.916227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:11.976382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ee1b-1c81-47d9-88bb-ee64fd0fb30d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:11.981307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:11.981993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:11.982343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:11.983273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:11.983800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:11.987347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:12.016995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 954/1896] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:11 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:12.021051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af8b1-2366-4c21-8b13-947961e86b55 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:12.033420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af8b1-2366-4c21-8b13-947961e86b55 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:12.097215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:12.098139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:12.098751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:12.099180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:12.100104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:12.102784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:12.118340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:12.134063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 943/1897] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:12 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:12.137896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:12.153418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965583fe-b3d2-4709-8c6c-f13bbf125b04 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:12.160900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:12.162614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:12.162899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:12.163304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:12.164213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:12.166562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:12.184033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 955/1898] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:12.185663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:12.199437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283af67-c1ad-43f3-8df8-687f53bd6a84 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:12.205550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:12.207327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:12.207327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:12.207327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] PATH_INFO: `/v3/projects/5ccfc6545eb2460fb9da04e813b3b88c/users/7458cf1683fe45c9874d31a7905513ea/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:12.241469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccfc6545eb2460fb9da04e813b3b88c, user_id=7458cf1683fe45c9874d31a7905513ea, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:12.244155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:12.258850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 944/1899] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:12 2022] PUT /identity/v3/projects/5ccfc6545eb2460fb9da04e813b3b88c/users/7458cf1683fe45c9874d31a7905513ea/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:12.262867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:12.273475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d20801-0351-4182-b714-9132161756f5 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:12.336885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe8742-73c3-4073-8dcc-273bae6cd5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:12.337608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe8742-73c3-4073-8dcc-273bae6cd5ff admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:12.337945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe8742-73c3-4073-8dcc-273bae6cd5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:12.338276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe8742-73c3-4073-8dcc-273bae6cd5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:12.338981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe8742-73c3-4073-8dcc-273bae6cd5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:12.340706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe8742-73c3-4073-8dcc-273bae6cd5ff admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:12.351592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 956/1900] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:12.359459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:12.366520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56ca8-b979-4e89-a3cb-21b83648e1b3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:12.372601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:12.373369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:12.373650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:12.373924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] PATH_INFO: `/v3/projects/5ccfc6545eb2460fb9da04e813b3b88c/users/7458cf1683fe45c9874d31a7905513ea/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:12.375937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccfc6545eb2460fb9da04e813b3b88c, user_id=7458cf1683fe45c9874d31a7905513ea, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:12.378846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:12.389235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 945/1901] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:12 2022] PUT /identity/v3/projects/5ccfc6545eb2460fb9da04e813b3b88c/users/7458cf1683fe45c9874d31a7905513ea/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:12.400800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:12.401089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:12.401366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:12.437579 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] MFA Rules not processed for user `7458cf1683fe45c9874d31a7905513ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:12.438915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:12.506976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:12.510876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 957/1902] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:12.523111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:12.525061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:12.589653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a108be4-ab7d-4706-bf65-2c9db9e1e471 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:12.596453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:12.597102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:12.597375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:12.597642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:12.662773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:12.665073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:12.677438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 946/1903] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:16.507295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:16.509093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:16.521634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021985a8-7648-4cab-ae0f-82ce86704619 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:16.526128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:16.526802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:16.527144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:16.527458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:16.592073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:16.593893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:16.603145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 958/1904] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:17.577537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:17.586499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ecb4e-2296-4d24-9e2a-d2773ce8cbad service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:17.590917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaac6f-d9c1-45af-98a2-2e3872ce6543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:17.591178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaac6f-d9c1-45af-98a2-2e3872ce6543 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:17.591456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaac6f-d9c1-45af-98a2-2e3872ce6543 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:17.591758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaac6f-d9c1-45af-98a2-2e3872ce6543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:17.603614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baaac6f-d9c1-45af-98a2-2e3872ce6543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:17.605134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baaac6f-d9c1-45af-98a2-2e3872ce6543 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:17.616706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 947/1905] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:17.909341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:17.910807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:17.977648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c382a0-8f55-4a07-bbc2-6cefed27b042 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:17.982518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:17.983338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:17.983506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:17.983841 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] PATH_INFO: `/v3/users/0eaaf3b575944a8c98e2311b92e216be` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:17.984975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eaaf3b575944a8c98e2311b92e216be)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:17.987004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:18.041865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eaaf3b575944a8c98e2311b92e216be'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.042506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eaaf3b575944a8c98e2311b92e216be'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.049115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eaaf3b575944a8c98e2311b92e216be'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.050080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eaaf3b575944a8c98e2311b92e216be'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.058415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eaaf3b575944a8c98e2311b92e216be'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.076066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 959/1906] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:17 2022] DELETE /identity/v3/users/0eaaf3b575944a8c98e2311b92e216be => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:18.081798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:18.082081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:18.082349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:18.102304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:18.103209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:18.168772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:18.184030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 948/1907] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:18.201645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:18.203440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:18.227139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:18.268355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd036-c3e3-4b4d-a156-84ed52752084 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:18.272117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:18.272738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:18.273019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:18.273294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:18.282211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:18.283805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:18.293463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 960/1908] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:18.675850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:18.677447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:18.691916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e9b08-09ac-4c93-b7cf-5bcd33b48bac None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:18.696805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:18.697468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:18.697751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:18.698029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] PATH_INFO: `/v3/projects/d9f882b621c44f928493f9afda0b9248` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:18.699164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f882b621c44f928493f9afda0b9248)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:18.701124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:18.749055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f882b621c44f928493f9afda0b9248', 'request_id': 'req-5b092e17-3750-4565-9f4f-b1943762b7a9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.754476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f882b621c44f928493f9afda0b9248', 'request_id': 'req-5b092e17-3750-4565-9f4f-b1943762b7a9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.760706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f882b621c44f928493f9afda0b9248', 'request_id': 'req-5b092e17-3750-4565-9f4f-b1943762b7a9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.775552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] The token cache is being invalidate because project d9f882b621c44f928493f9afda0b9248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:18.775927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.776628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.778731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 949/1909] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:18 2022] DELETE /identity/v3/projects/d9f882b621c44f928493f9afda0b9248 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:18.785091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:18.802964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60590f-43c9-4c06-b6cb-b489d50a64cc service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:18.886569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:18.887309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:18.887693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:18.888064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] PATH_INFO: `/v3/users/0d96ca77475c44559746e312b3469cbf` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:18.889218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d96ca77475c44559746e312b3469cbf)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:18.891176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:18.942939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d96ca77475c44559746e312b3469cbf'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.943631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d96ca77475c44559746e312b3469cbf'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.957011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d96ca77475c44559746e312b3469cbf'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.957707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d96ca77475c44559746e312b3469cbf'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.968404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-afbf7dfb-ce04-4627-9308-017650029291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d96ca77475c44559746e312b3469cbf'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:18.986378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 961/1910] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:18 2022] DELETE /identity/v3/users/0d96ca77475c44559746e312b3469cbf => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:19.274974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:19.276669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:19.291075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b092e17-3750-4565-9f4f-b1943762b7a9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:19.357390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:19.358039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.358318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.358609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] PATH_INFO: `/v3/projects/5c1c39ae6843448781cbbbfd5a0e5fdd` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.359753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c1c39ae6843448781cbbbfd5a0e5fdd)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:19.361676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:19.409778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c1c39ae6843448781cbbbfd5a0e5fdd', 'request_id': 'req-bfca63a9-4cd7-48be-a493-ed79e182bd7c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:19.415906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c1c39ae6843448781cbbbfd5a0e5fdd', 'request_id': 'req-bfca63a9-4cd7-48be-a493-ed79e182bd7c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:19.422199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c1c39ae6843448781cbbbfd5a0e5fdd', 'request_id': 'req-bfca63a9-4cd7-48be-a493-ed79e182bd7c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:19.435847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] The token cache is being invalidate because project 5c1c39ae6843448781cbbbfd5a0e5fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:19.436137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:19.436755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bfca63a9-4cd7-48be-a493-ed79e182bd7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:19.438331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 950/1911] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:19 2022] DELETE /identity/v3/projects/5c1c39ae6843448781cbbbfd5a0e5fdd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:19.444023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b5260-306a-4cc2-9d7a-4c0dfd6e8ef5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.444307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b5260-306a-4cc2-9d7a-4c0dfd6e8ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.444581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b5260-306a-4cc2-9d7a-4c0dfd6e8ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.464767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b58b5260-306a-4cc2-9d7a-4c0dfd6e8ef5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:19.466252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b58b5260-306a-4cc2-9d7a-4c0dfd6e8ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:19.504364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:19.534415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 962/1912] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:19.542345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a05e27-bcb6-4e4d-a1b3-665678f380e7 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.542638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a05e27-bcb6-4e4d-a1b3-665678f380e7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.542914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a05e27-bcb6-4e4d-a1b3-665678f380e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.572151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-30a05e27-bcb6-4e4d-a1b3-665678f380e7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:19.573397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-30a05e27-bcb6-4e4d-a1b3-665678f380e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:19.581801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 951/1913] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:19.586198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b5260-306a-4cc2-9d7a-4c0dfd6e8ef5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:19.596483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b5260-306a-4cc2-9d7a-4c0dfd6e8ef5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:19.601244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:19.601862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.602219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.602494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.603151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:19.603457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:19.605213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:19.621002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 963/1914] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:19.622034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a05e27-bcb6-4e4d-a1b3-665678f380e7 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:19.643636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a05e27-bcb6-4e4d-a1b3-665678f380e7 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:19.643636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:19.643636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.643636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.644596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.644596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:19.644596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:19.649713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:19.687674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 952/1915] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:19 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:19.689449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:19.704189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565dd91-34f9-49b3-8d0f-4f7f910721f4 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:19.784837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:19.785491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.785774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.786047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.786687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:19.788724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:19.811680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 964/1916] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:19 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:19.816244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:19.828859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c1dae-db97-45b5-9084-8f9b79e1c258 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:19.834234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:19.834900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.835520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.835520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.836135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:19.837933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:19.853634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 953/1917] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:19.854902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:19.871693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4c18d-9df5-433f-8abf-d5fd344fc9f2 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:19.877332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd306705-d88f-4feb-bdcf-98704e9689fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:19.878338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306705-d88f-4feb-bdcf-98704e9689fc admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.878718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306705-d88f-4feb-bdcf-98704e9689fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.879089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306705-d88f-4feb-bdcf-98704e9689fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.879966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd306705-d88f-4feb-bdcf-98704e9689fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:19.888876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd306705-d88f-4feb-bdcf-98704e9689fc admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:19.919085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 965/1918] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:19.923483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:19.934852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3493a14-4faa-474f-842a-d92a08de29d4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:19.938955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:19.939725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.940063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.940497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] PATH_INFO: `/v3/projects/646a0f6383bc4392807aa97c71d8638c/users/6e4c91e49a3b4e52a5ac90b5a3df31a0/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:19.955377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646a0f6383bc4392807aa97c71d8638c, user_id=6e4c91e49a3b4e52a5ac90b5a3df31a0, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:19.958083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:19.977692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 954/1919] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:19 2022] PUT /identity/v3/projects/646a0f6383bc4392807aa97c71d8638c/users/6e4c91e49a3b4e52a5ac90b5a3df31a0/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:19.989853 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:19.990725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:19.991005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:20.021732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] MFA Rules not processed for user `6e4c91e49a3b4e52a5ac90b5a3df31a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:20.023070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:20.108888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 966/1920] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:20.120405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:20.122209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:20.197369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751b06-ae48-4ca4-9fb5-d97b580faa80 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:20.201750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34091897-6be5-485b-b82d-38c77dca1d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:20.202376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091897-6be5-485b-b82d-38c77dca1d77 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:20.202654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091897-6be5-485b-b82d-38c77dca1d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:20.202945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091897-6be5-485b-b82d-38c77dca1d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:20.212391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34091897-6be5-485b-b82d-38c77dca1d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:20.213832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34091897-6be5-485b-b82d-38c77dca1d77 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:20.224303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 955/1921] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:30.105115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:30.219185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:30.645683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:30.648065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:30.737887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d2b69-1ab7-497a-8eee-3e9502fa4b22 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:30.743782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:30.744442 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:30.744764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:30.745042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] PATH_INFO: `/v3/users/f519ba53264140098982d2ca6877a4b8` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:30.746375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f519ba53264140098982d2ca6877a4b8)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:30.748665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:30.801897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f519ba53264140098982d2ca6877a4b8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:30.802602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f519ba53264140098982d2ca6877a4b8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:30.809484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f519ba53264140098982d2ca6877a4b8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:30.810097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f519ba53264140098982d2ca6877a4b8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:30.817119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f519ba53264140098982d2ca6877a4b8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:30.834270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 967/1922] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:30 2022] DELETE /identity/v3/users/f519ba53264140098982d2ca6877a4b8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:30.839080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:30.839362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:30.839663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:30.860769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:30.862006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:30.930911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 956/1923] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:30.940902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:30.942693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:31.025781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4b6b5-32ab-4ac8-9116-b28dc0fbd688 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:31.029669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ad94e-5947-4ec0-85ed-11ee95432d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:31.030682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ad94e-5947-4ec0-85ed-11ee95432d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:31.030682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ad94e-5947-4ec0-85ed-11ee95432d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:31.030891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ad94e-5947-4ec0-85ed-11ee95432d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:31.041625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ad94e-5947-4ec0-85ed-11ee95432d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:31.043093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ad94e-5947-4ec0-85ed-11ee95432d46 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:31.053698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 968/1924] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:31.294300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:31.295827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:31.309433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54329e0-898f-45b5-a544-86f75be1c0a3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:31.314001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:31.314663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:31.314943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:31.315215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] PATH_INFO: `/v3/users/2f70a12b43884d92abfbaf9167e35c88` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:31.316378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f70a12b43884d92abfbaf9167e35c88)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:31.318441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:31.375283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f70a12b43884d92abfbaf9167e35c88'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.376051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f70a12b43884d92abfbaf9167e35c88'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.376655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f70a12b43884d92abfbaf9167e35c88'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.388085 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f70a12b43884d92abfbaf9167e35c88'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.396490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f70a12b43884d92abfbaf9167e35c88'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.427984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 957/1925] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:31 2022] DELETE /identity/v3/users/2f70a12b43884d92abfbaf9167e35c88 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:31.432766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:31.433263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:31.433263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:31.453404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:31.454634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:31.475091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:31.528324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 969/1926] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:31.542560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:31.544115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:31.606073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d7e33-b636-4f7c-8a35-7f011f2604b4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:31.612573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:31.612855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:31.614082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:31.614197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:31.614659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:31.615086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:31.626332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:31.626641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949ce3c-6254-452e-b6b8-f56f1099afe3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:31.628044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:31.632008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:31.632665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:31.633005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:31.633245 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] PATH_INFO: `/v3/projects/76f2673913814b65870d97a33966444c` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:31.634390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f2673913814b65870d97a33966444c)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:31.636371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:31.641471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 958/1927] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:31.686778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f2673913814b65870d97a33966444c', 'request_id': 'req-0c2a0c46-3c0a-481f-9283-32d30fad1996'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.692941 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f2673913814b65870d97a33966444c', 'request_id': 'req-0c2a0c46-3c0a-481f-9283-32d30fad1996'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.699657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f2673913814b65870d97a33966444c', 'request_id': 'req-0c2a0c46-3c0a-481f-9283-32d30fad1996'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.712921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] The token cache is being invalidate because project 76f2673913814b65870d97a33966444c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:31.713209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.713910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.715524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 970/1928] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:31 2022] DELETE /identity/v3/projects/76f2673913814b65870d97a33966444c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:31.720106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:31.730459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d242b-8463-449a-bb8c-6968a1e59a43 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:31.742399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:31.802619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:31.803473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:31.803473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:31.803805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] PATH_INFO: `/v3/users/e608cf02112548b2a4578dd9454e1f6c` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:31.804937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e608cf02112548b2a4578dd9454e1f6c)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:31.806824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:31.856166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e608cf02112548b2a4578dd9454e1f6c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.856933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e608cf02112548b2a4578dd9454e1f6c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.857616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e608cf02112548b2a4578dd9454e1f6c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.866268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e608cf02112548b2a4578dd9454e1f6c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.873762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e608cf02112548b2a4578dd9454e1f6c'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:31.891655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 959/1929] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:31 2022] DELETE /identity/v3/users/e608cf02112548b2a4578dd9454e1f6c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:31.927527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:31.929021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:31.946241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a0c46-3c0a-481f-9283-32d30fad1996 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.008055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.008708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.008988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.009261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] PATH_INFO: `/v3/projects/79318ba980164cc8bd4e516cd55886c6` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.010400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79318ba980164cc8bd4e516cd55886c6)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.012352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.061247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:32.062931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79318ba980164cc8bd4e516cd55886c6', 'request_id': 'req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.070384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79318ba980164cc8bd4e516cd55886c6', 'request_id': 'req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.076576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79318ba980164cc8bd4e516cd55886c6', 'request_id': 'req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.090324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] The token cache is being invalidate because project 79318ba980164cc8bd4e516cd55886c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:32.090606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.091227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.093294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 971/1930] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:31 2022] DELETE /identity/v3/projects/79318ba980164cc8bd4e516cd55886c6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:32.097397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.099099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.112360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26fa49-2305-4791-9254-f015338f5461 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.186516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.189091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.192476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.193139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.193421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.193702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] PATH_INFO: `/v3/users/5d0652150115447ab08449d61bf05389` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.194866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0652150115447ab08449d61bf05389)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.197041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.223188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbeaf1c-6ee6-45f7-9b4a-a0cd3497a017 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.228563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.229279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.229561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.229865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] PATH_INFO: `/v3/projects/bd6d9c79f4ca46e080b521f48b658297` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.231064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6d9c79f4ca46e080b521f48b658297)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.233185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.254129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:32.265350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0652150115447ab08449d61bf05389'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.266049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0652150115447ab08449d61bf05389'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.266907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0652150115447ab08449d61bf05389'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.278116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0652150115447ab08449d61bf05389'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.289207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34235752-9dbc-43ca-b501-fc1a810bf71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0652150115447ab08449d61bf05389'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.298275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6d9c79f4ca46e080b521f48b658297', 'request_id': 'req-876226cc-b252-4fbf-980a-307f1ee1f3a8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.305318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6d9c79f4ca46e080b521f48b658297', 'request_id': 'req-876226cc-b252-4fbf-980a-307f1ee1f3a8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.310775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 960/1931] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:32 2022] DELETE /identity/v3/users/5d0652150115447ab08449d61bf05389 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:32.317222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6d9c79f4ca46e080b521f48b658297', 'request_id': 'req-876226cc-b252-4fbf-980a-307f1ee1f3a8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.337323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] The token cache is being invalidate because project bd6d9c79f4ca46e080b521f48b658297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:32.338380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.340936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-876226cc-b252-4fbf-980a-307f1ee1f3a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.343987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 972/1932] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:32 2022] DELETE /identity/v3/projects/bd6d9c79f4ca46e080b521f48b658297 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:32.349054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb88d7-8691-489d-8553-f4b99cb1b473 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.349374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb88d7-8691-489d-8553-f4b99cb1b473 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.350092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb88d7-8691-489d-8553-f4b99cb1b473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.377751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c3fb88d7-8691-489d-8553-f4b99cb1b473 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:32.379044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c3fb88d7-8691-489d-8553-f4b99cb1b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.450989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 961/1933] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:32.463291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb438e39-db47-4ef6-9cf6-fca459fa75ff None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.463855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb438e39-db47-4ef6-9cf6-fca459fa75ff None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.464012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb438e39-db47-4ef6-9cf6-fca459fa75ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.484938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-cb438e39-db47-4ef6-9cf6-fca459fa75ff None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:32.487892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cb438e39-db47-4ef6-9cf6-fca459fa75ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.503133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 973/1934] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:32.507932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb88d7-8691-489d-8553-f4b99cb1b473 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.518197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb88d7-8691-489d-8553-f4b99cb1b473 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.523371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.524032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.524313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.524601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.525242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:32.525544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.527255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.539407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 962/1935] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:32.543905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb438e39-db47-4ef6-9cf6-fca459fa75ff None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.555191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb438e39-db47-4ef6-9cf6-fca459fa75ff None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.561007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.561707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.562052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.562391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.563893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.566334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.602929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 974/1936] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:32.607267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.617790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86827be9-ca59-40e0-94c3-b213f11af7fb admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.649501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.651335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.692098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.692989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.693332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.693674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.694383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.696481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.724735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce9d56-32b4-49cf-bde2-81100dd75d7f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.731392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.731552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 963/1937] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:32.735181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.735471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.735786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] PATH_INFO: `/v3/projects/da9b2a0c65bd4e7697dafc9059973372` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.735883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.736936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9b2a0c65bd4e7697dafc9059973372)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.738970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.746264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ac95-4117-4eae-967b-b403d86cc04a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.750923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.751618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.751963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.752405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.753053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.754765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.765623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 964/1938] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:32.770000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.778034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:32.781991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29dc01-4a17-4586-8e59-d22148bf4b87 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.791922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9b2a0c65bd4e7697dafc9059973372', 'request_id': 'req-a0f71491-5811-4740-a47b-d0dd3b62662b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.797355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9b2a0c65bd4e7697dafc9059973372', 'request_id': 'req-a0f71491-5811-4740-a47b-d0dd3b62662b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.799811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:32.804670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9b2a0c65bd4e7697dafc9059973372', 'request_id': 'req-a0f71491-5811-4740-a47b-d0dd3b62662b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.818505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] The token cache is being invalidate because project da9b2a0c65bd4e7697dafc9059973372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:32.818797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.820563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a0f71491-5811-4740-a47b-d0dd3b62662b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:32.821319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 975/1939] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:32 2022] DELETE /identity/v3/projects/da9b2a0c65bd4e7697dafc9059973372 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:32.826129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141adff-7432-41c3-8d36-6dee52fc8fd3 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.826421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141adff-7432-41c3-8d36-6dee52fc8fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.826725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141adff-7432-41c3-8d36-6dee52fc8fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.849424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.850048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.850338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.850610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.851171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9141adff-7432-41c3-8d36-6dee52fc8fd3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:32.851285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.852695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9141adff-7432-41c3-8d36-6dee52fc8fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.852854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.866150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 965/1940] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:32.866279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 976/1941] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:32.867670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.869256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9bf1b-d1f7-4e19-bff4-c6ad0bf51178 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.869536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9bf1b-d1f7-4e19-bff4-c6ad0bf51178 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.869807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9bf1b-d1f7-4e19-bff4-c6ad0bf51178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.875438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.888195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66969423-2def-4cfe-8281-18b5b8a70535 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.890537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f8f9bf1b-d1f7-4e19-bff4-c6ad0bf51178 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:32.891860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f8f9bf1b-d1f7-4e19-bff4-c6ad0bf51178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:32.892948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2a476-47bf-485d-945e-7e9de98f4830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.893617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2a476-47bf-485d-945e-7e9de98f4830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.893951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2a476-47bf-485d-945e-7e9de98f4830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.894167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2a476-47bf-485d-945e-7e9de98f4830 admin admin] PATH_INFO: `/v3/projects/c3e02b9af7464fe8bafc576c3659a9e1/users/06c403d52d71414791df16f892d397e9/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.903005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 977/1942] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:32.907984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9bf1b-d1f7-4e19-bff4-c6ad0bf51178 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.911945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2a476-47bf-485d-945e-7e9de98f4830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e02b9af7464fe8bafc576c3659a9e1, user_id=06c403d52d71414791df16f892d397e9, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.914593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2a476-47bf-485d-945e-7e9de98f4830 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.917141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9bf1b-d1f7-4e19-bff4-c6ad0bf51178 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.921793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:32.922377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.922652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.922918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.923590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:32.923892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:32.925481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:32.928690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 966/1943] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:32 2022] PUT /identity/v3/projects/c3e02b9af7464fe8bafc576c3659a9e1/users/06c403d52d71414791df16f892d397e9/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:32.937826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 978/1944] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:32.939253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477187a-80d8-413d-b11b-ce9ce1b382b1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:32.939543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477187a-80d8-413d-b11b-ce9ce1b382b1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:32.939852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477187a-80d8-413d-b11b-ce9ce1b382b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:32.941811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:32.951825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712685f-c757-4553-b77a-65073510e50f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:32.967386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6477187a-80d8-413d-b11b-ce9ce1b382b1 None None] MFA Rules not processed for user `06c403d52d71414791df16f892d397e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:32.969001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6477187a-80d8-413d-b11b-ce9ce1b382b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:33.011362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.012013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.012283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.012554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.013193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.015112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.033605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 967/1945] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:33.038523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477187a-80d8-413d-b11b-ce9ce1b382b1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.043558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 979/1946] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:32 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:33.047796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.049780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477187a-80d8-413d-b11b-ce9ce1b382b1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.106953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.107596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.107891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.108160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.108807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.110085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9c1a4-4c6c-4c09-a069-2ec99ba2ef0a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.110702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.115226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.115878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.116159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.116432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.117061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.118990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.139776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 968/1947] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:33.144276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 980/1948] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:33.145703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.145825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.154261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:33.156705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7561a8-a66a-414c-a29a-90eba8efa2ad admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.162328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:33.209256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e486-8bd4-4701-ac39-8d9534b151a2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.214013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.214630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.214910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.215179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.215841 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.216741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.217356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.217631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.218145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.218381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.219029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.220730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.231174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 981/1949] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:33.235203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.242894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 969/1950] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:33.244107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.245533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee418256-c37f-4cc7-b990-3e43b9c04ab0 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.249952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.250550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.250824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.251093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.251798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.253423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.253708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485c24-47ae-4f3a-9d4c-5de5ed382915 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.258439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.259054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.259332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.259637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.260285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.261865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.264128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 982/1951] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:33.268156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.276234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 970/1952] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:33.277258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.278060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2fa5-7ff9-43ed-af16-3676586d2624 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.282575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7488de-4e0d-4c14-aee6-6cee8232cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.283225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7488de-4e0d-4c14-aee6-6cee8232cac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.283504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7488de-4e0d-4c14-aee6-6cee8232cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.283809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7488de-4e0d-4c14-aee6-6cee8232cac6 admin admin] PATH_INFO: `/v3/projects/8228b71fa8aa481cbfb604bf37483827/users/751366be0dba494f86dd3dc62cfc6b6f/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.287154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091e8de-5913-4e0e-890e-e103aca18a80 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.291833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.292506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.292784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.293056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] PATH_INFO: `/v3/projects/4a54457d6f9a41d4bfa1f4b3759b09a9/users/f4006848f77e44279effddec06be8183/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.299278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7488de-4e0d-4c14-aee6-6cee8232cac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8228b71fa8aa481cbfb604bf37483827, user_id=751366be0dba494f86dd3dc62cfc6b6f, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.302331 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7488de-4e0d-4c14-aee6-6cee8232cac6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.307774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a54457d6f9a41d4bfa1f4b3759b09a9, user_id=f4006848f77e44279effddec06be8183, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.310610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.316324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 983/1953] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:33 2022] PUT /identity/v3/projects/8228b71fa8aa481cbfb604bf37483827/users/751366be0dba494f86dd3dc62cfc6b6f/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:33.324125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 971/1954] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:33 2022] PUT /identity/v3/projects/4a54457d6f9a41d4bfa1f4b3759b09a9/users/f4006848f77e44279effddec06be8183/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:33.328116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b133-0b2b-4d5d-89bc-243468878931 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.328402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b133-0b2b-4d5d-89bc-243468878931 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.328670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.328779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b133-0b2b-4d5d-89bc-243468878931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.359990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-ef50b133-0b2b-4d5d-89bc-243468878931 None None] MFA Rules not processed for user `751366be0dba494f86dd3dc62cfc6b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:33.361758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ef50b133-0b2b-4d5d-89bc-243468878931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:33.427448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871e6a-2161-4fb8-9df0-b114752ab471 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.435165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.435939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.436227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.436941 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.438594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.440402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.459394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 972/1955] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:33.461198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 984/1956] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:33.464496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.465471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b133-0b2b-4d5d-89bc-243468878931 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.477350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3196d-bca8-4c9b-950e-564bba3ff213 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.477553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b133-0b2b-4d5d-89bc-243468878931 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.482865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18a1ca-e25b-4d0f-9462-243012005a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.484026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18a1ca-e25b-4d0f-9462-243012005a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.484306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18a1ca-e25b-4d0f-9462-243012005a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.484575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18a1ca-e25b-4d0f-9462-243012005a1a admin admin] PATH_INFO: `/v3/projects/4a54457d6f9a41d4bfa1f4b3759b09a9/users/f4006848f77e44279effddec06be8183/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.488598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.489109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18a1ca-e25b-4d0f-9462-243012005a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a54457d6f9a41d4bfa1f4b3759b09a9, user_id=f4006848f77e44279effddec06be8183, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.489877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.490318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.491393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.491975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18a1ca-e25b-4d0f-9462-243012005a1a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.492392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.496380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.503646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 973/1957] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:33 2022] PUT /identity/v3/projects/4a54457d6f9a41d4bfa1f4b3759b09a9/users/f4006848f77e44279effddec06be8183/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:33.514940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.515219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.515488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.540811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 985/1958] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:33.542772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.556895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] MFA Rules not processed for user `f4006848f77e44279effddec06be8183`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:33.558156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:33.599252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:33.606341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:33.631597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6f400-969f-424e-a3b8-ff317abbfcf0 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.634293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.635037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.635379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.635746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.636401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.638315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.644506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 974/1959] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:33.656834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.658318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:33.660578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 986/1960] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:33.664343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.674510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5be6-0122-4dd4-9e3f-e4f9c70c3aef admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.679642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.680250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.680527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.680794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.681438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.683037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.693640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 987/1961] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:33.697804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.707644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58927926-aacc-4a04-850b-5324d098f853 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.712440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.713028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.713310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.713588 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] PATH_INFO: `/v3/projects/9c4266e041fd4ac7b33d0297c209d984/users/f5f42ecf1af34430a973fb4109c06302/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.723308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955b6fe-38b6-432c-af44-6a13d40c99fd None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.726952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.727563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.727862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.728132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.729273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4266e041fd4ac7b33d0297c209d984, user_id=f5f42ecf1af34430a973fb4109c06302, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.732081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.746209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.750894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 988/1962] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:33 2022] PUT /identity/v3/projects/9c4266e041fd4ac7b33d0297c209d984/users/f5f42ecf1af34430a973fb4109c06302/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:33.751032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.752105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.759019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 975/1963] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:33.815215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.817492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:33.827927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabcece-b988-4c0e-a288-dae4feb3729d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.829330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bb008-6c77-4348-a626-00496809066c service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.833248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.833868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.834148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.834423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.835211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.836915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.848559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 989/1964] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:33.853094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:33.863211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a7b11-d3f6-4fff-94e1-3c2e6fb2564c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:33.870975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29098b89-3ab2-402d-a388-896359eaed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.871817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29098b89-3ab2-402d-a388-896359eaed8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.872164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29098b89-3ab2-402d-a388-896359eaed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.872551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29098b89-3ab2-402d-a388-896359eaed8d admin admin] PATH_INFO: `/v3/projects/9c4266e041fd4ac7b33d0297c209d984/users/f5f42ecf1af34430a973fb4109c06302/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.874907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29098b89-3ab2-402d-a388-896359eaed8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4266e041fd4ac7b33d0297c209d984, user_id=f5f42ecf1af34430a973fb4109c06302, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.877517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29098b89-3ab2-402d-a388-896359eaed8d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.890788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:33.891423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.891819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.892091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.899188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 990/1965] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:33 2022] PUT /identity/v3/projects/9c4266e041fd4ac7b33d0297c209d984/users/f5f42ecf1af34430a973fb4109c06302/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:33.900960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafc857-146f-4f76-a62d-1312beb47e4c None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:33.901346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafc857-146f-4f76-a62d-1312beb47e4c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:33.901713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafc857-146f-4f76-a62d-1312beb47e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:33.949284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6eafc857-146f-4f76-a62d-1312beb47e4c None None] MFA Rules not processed for user `f5f42ecf1af34430a973fb4109c06302`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:33.951054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6eafc857-146f-4f76-a62d-1312beb47e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:33.966962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:33.968719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:33.980333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 976/1966] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:39:34.021432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:34.028318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 991/1967] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:34.037074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:34.045805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9790b-84c8-428d-ab68-ea59562827d4 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:34.072904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:34.097628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:34.098399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:34.098676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:34.098944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:34.108374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:34.109807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:34.120268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 977/1968] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:34.204694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafc857-146f-4f76-a62d-1312beb47e4c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:34.214888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafc857-146f-4f76-a62d-1312beb47e4c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:34.219035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:34.219725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:34.220059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:34.220238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:34.300626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:34.302958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:34.313282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 992/1969] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:37.738691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:37.791934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:37.909392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f9b6c-c914-4976-bc5c-1dc75f028433 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:37.915132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:37.916736 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:37.917203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:37.917656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] PATH_INFO: `/v3/users/06c403d52d71414791df16f892d397e9` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:37.919435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c403d52d71414791df16f892d397e9)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:37.920293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36b4f3-b924-41e1-a31d-c6489dfb596b service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:37.922164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:37.925093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:37.925740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:37.926021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:37.926294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] PATH_INFO: `/v3/users/751366be0dba494f86dd3dc62cfc6b6f` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:37.927984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=751366be0dba494f86dd3dc62cfc6b6f)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:37.929761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:37.992532 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c403d52d71414791df16f892d397e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:37.993074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c403d52d71414791df16f892d397e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:37.994522 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c403d52d71414791df16f892d397e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:37.996874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751366be0dba494f86dd3dc62cfc6b6f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:37.998257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751366be0dba494f86dd3dc62cfc6b6f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.003886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c403d52d71414791df16f892d397e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.012507 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751366be0dba494f86dd3dc62cfc6b6f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.013600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751366be0dba494f86dd3dc62cfc6b6f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.014251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c403d52d71414791df16f892d397e9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.038399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1a82870a-012c-4b23-8f04-3420cf3ac390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751366be0dba494f86dd3dc62cfc6b6f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.043151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 978/1970] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:37 2022] DELETE /identity/v3/users/06c403d52d71414791df16f892d397e9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:38.043151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:38.049315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.077448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 993/1971] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:37 2022] DELETE /identity/v3/users/751366be0dba494f86dd3dc62cfc6b6f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:38.077448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a6b3e-f7ad-48c4-9fbc-511c620d59cb None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.077448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a6b3e-f7ad-48c4-9fbc-511c620d59cb None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.077448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a6b3e-f7ad-48c4-9fbc-511c620d59cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.101277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-df3a6b3e-f7ad-48c4-9fbc-511c620d59cb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:38.102819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-df3a6b3e-f7ad-48c4-9fbc-511c620d59cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.118765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:38.128834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:38.157065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd801f3b-c53c-440f-8fb6-c961ce42bc00 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:38.175884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 994/1972] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:38.179485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.179485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.179485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.204890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:38.206170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.215090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 995/1973] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:38.219875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:38.221448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.256677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:38.257344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.257621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.257892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] PATH_INFO: `/v3/users/2d5f18eb43214ab68482ba8b0f7deeb5` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.260915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5f18eb43214ab68482ba8b0f7deeb5)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:38.262418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:38.315666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8667ca-fe2d-4916-be30-46a794b8c635 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:38.321690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:38.322356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.322644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.322918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.334645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:38.336209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:38.347600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 996/1974] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:38.350266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5f18eb43214ab68482ba8b0f7deeb5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.351168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5f18eb43214ab68482ba8b0f7deeb5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.353256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5f18eb43214ab68482ba8b0f7deeb5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.356302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:38.371086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5f18eb43214ab68482ba8b0f7deeb5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.381789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.384229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d200bc37-50b5-4a59-a56b-cb2e30e8dfee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5f18eb43214ab68482ba8b0f7deeb5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.401889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83130f-d3a7-4fa5-babb-53c4ae5f8241 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:38.406755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:38.407527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.407892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.409084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.413271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 979/1975] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:38 2022] DELETE /identity/v3/users/2d5f18eb43214ab68482ba8b0f7deeb5 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:38.413412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.415547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.415863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.416135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.437913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:38.440732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:38.446576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:38.446576 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.460497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 997/1976] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:38.569372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 980/1977] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:38.591411 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:38.593580 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.607062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a76a-4817-49b9-ab6c-5cc4ab38d03e service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:38.709860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:38.710599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.710964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.711307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.723706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:38.725472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:38.737793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 998/1978] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:38.860427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:38.862038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.875392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c760b-e79b-4b84-a8a0-ad49fa06b860 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:38.881103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:38.881769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:38.882217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:38.882494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] PATH_INFO: `/v3/projects/c3e02b9af7464fe8bafc576c3659a9e1` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:38.883702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e02b9af7464fe8bafc576c3659a9e1)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:38.886288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:38.917081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:38.942233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:38.943943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:38.957572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae838f4-ef7b-4e18-8532-397014a3906a service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:38.976454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e02b9af7464fe8bafc576c3659a9e1', 'request_id': 'req-25da634b-65c8-4b22-93f3-70344a4742c7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.984631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e02b9af7464fe8bafc576c3659a9e1', 'request_id': 'req-25da634b-65c8-4b22-93f3-70344a4742c7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:38.992465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e02b9af7464fe8bafc576c3659a9e1', 'request_id': 'req-25da634b-65c8-4b22-93f3-70344a4742c7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.003785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:39.007815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] The token cache is being invalidate because project c3e02b9af7464fe8bafc576c3659a9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:39.008142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.009481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.011216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 981/1979] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:38 2022] DELETE /identity/v3/projects/c3e02b9af7464fe8bafc576c3659a9e1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:39.015950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:39.017345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.023352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:39.024236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.024346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.024573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] PATH_INFO: `/v3/users/6e4c91e49a3b4e52a5ac90b5a3df31a0` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.026582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4c91e49a3b4e52a5ac90b5a3df31a0)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:39.029197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:39.031203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da634b-65c8-4b22-93f3-70344a4742c7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:39.036923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:39.037129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.037310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.037540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] PATH_INFO: `/v3/users/f4006848f77e44279effddec06be8183` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.038965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4006848f77e44279effddec06be8183)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:39.040640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:39.119260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4006848f77e44279effddec06be8183'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.119954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4006848f77e44279effddec06be8183'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.120917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4006848f77e44279effddec06be8183'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.130505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4c91e49a3b4e52a5ac90b5a3df31a0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.131453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4c91e49a3b4e52a5ac90b5a3df31a0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.133086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4006848f77e44279effddec06be8183'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.141781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4c91e49a3b4e52a5ac90b5a3df31a0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.142223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4006848f77e44279effddec06be8183'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.142422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4c91e49a3b4e52a5ac90b5a3df31a0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.155164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fe1107b6-efbe-4460-b0c4-bd26d0185a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4c91e49a3b4e52a5ac90b5a3df31a0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.167934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 982/1980] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:39 2022] DELETE /identity/v3/users/f4006848f77e44279effddec06be8183 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:39.168673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:39.170184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.185291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 999/1981] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:38 2022] DELETE /identity/v3/users/6e4c91e49a3b4e52a5ac90b5a3df31a0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:39.189991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.190486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.190961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.226309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:39.231950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.290680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08369574-845e-4723-a2c7-661c58d9d48d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:39.314546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:39.326281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1000/1982] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:39.326959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:39.339701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.359715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770de5c-19cc-4ee0-b35b-13ae99d10475 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:39.366580 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:39.367758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.368140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.368486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] PATH_INFO: `/v3/projects/23b4e95b667e46e9a1556ec7a6d8b113` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.370007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b4e95b667e46e9a1556ec7a6d8b113)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:39.372420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:39.382799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:39.383972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.384456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.384894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] PATH_INFO: `/v3/projects/8228b71fa8aa481cbfb604bf37483827` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.386611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8228b71fa8aa481cbfb604bf37483827)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:39.388832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:39.448846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b4e95b667e46e9a1556ec7a6d8b113', 'request_id': 'req-384e5d3d-2538-46dc-be88-501cde8e7195'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.475798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b4e95b667e46e9a1556ec7a6d8b113', 'request_id': 'req-384e5d3d-2538-46dc-be88-501cde8e7195'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.477253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:39.482034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8228b71fa8aa481cbfb604bf37483827', 'request_id': 'req-ffbde71c-3f70-4d7c-bcde-4545e3962223'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.488030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8228b71fa8aa481cbfb604bf37483827', 'request_id': 'req-ffbde71c-3f70-4d7c-bcde-4545e3962223'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.494715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8228b71fa8aa481cbfb604bf37483827', 'request_id': 'req-ffbde71c-3f70-4d7c-bcde-4545e3962223'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.494929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b4e95b667e46e9a1556ec7a6d8b113', 'request_id': 'req-384e5d3d-2538-46dc-be88-501cde8e7195'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.510837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] The token cache is being invalidate because project 8228b71fa8aa481cbfb604bf37483827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:39.511133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.511876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.515609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 983/1983] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:39 2022] DELETE /identity/v3/projects/8228b71fa8aa481cbfb604bf37483827 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:39.515609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:39.518238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.523246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] The token cache is being invalidate because project 23b4e95b667e46e9a1556ec7a6d8b113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:39.523541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.524294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.525988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1001/1984] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:39 2022] DELETE /identity/v3/projects/23b4e95b667e46e9a1556ec7a6d8b113 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:39.527445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:39.530538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.607167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde71c-3f70-4d7c-bcde-4545e3962223 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:39.611608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:39.612284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.612654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.612933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.615661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.618702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e5d3d-2538-46dc-be88-501cde8e7195 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:39.700093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:39.700745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.701033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.701295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] PATH_INFO: `/v3/users/f5f42ecf1af34430a973fb4109c06302` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.702481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5f42ecf1af34430a973fb4109c06302)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:39.704469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:39.714578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:39.716073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:39.720779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:39.727995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 984/1985] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:39.728932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:39.731290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.747903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ae901-25ed-4d62-bffb-a17006efc70d service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:39.759204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:39.761011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:39.761807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:39.762239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] PATH_INFO: `/v3/users/5ff89cd266ba4173936412aa08dee3f3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:39.764863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff89cd266ba4173936412aa08dee3f3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:39.767012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:39.810312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5f42ecf1af34430a973fb4109c06302'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.811625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5f42ecf1af34430a973fb4109c06302'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.835925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5f42ecf1af34430a973fb4109c06302'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.836716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5f42ecf1af34430a973fb4109c06302'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.846652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff89cd266ba4173936412aa08dee3f3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.847423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff89cd266ba4173936412aa08dee3f3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.848186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff89cd266ba4173936412aa08dee3f3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.849854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5f42ecf1af34430a973fb4109c06302'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.859889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff89cd266ba4173936412aa08dee3f3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.872457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-2179b8f7-4211-4d4f-8ce2-6579cd7d79a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff89cd266ba4173936412aa08dee3f3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:39.877196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1002/1986] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:39 2022] DELETE /identity/v3/users/f5f42ecf1af34430a973fb4109c06302 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:39.878775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:39.881714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:39.907641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:39.926900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 985/1987] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:39 2022] DELETE /identity/v3/users/5ff89cd266ba4173936412aa08dee3f3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:40.034977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f85aa-05e1-4230-94ff-f54914f03299 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:40.143086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:40.143934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.144239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.144586 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] PATH_INFO: `/v3/projects/4a54457d6f9a41d4bfa1f4b3759b09a9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.145955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a54457d6f9a41d4bfa1f4b3759b09a9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:40.148327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:40.206472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a54457d6f9a41d4bfa1f4b3759b09a9', 'request_id': 'req-08678990-f335-4dd1-ae26-b77a8d198702'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.213439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a54457d6f9a41d4bfa1f4b3759b09a9', 'request_id': 'req-08678990-f335-4dd1-ae26-b77a8d198702'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.220534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a54457d6f9a41d4bfa1f4b3759b09a9', 'request_id': 'req-08678990-f335-4dd1-ae26-b77a8d198702'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.237533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] The token cache is being invalidate because project 4a54457d6f9a41d4bfa1f4b3759b09a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:40.237699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.238452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.240056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1003/1988] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:39 2022] DELETE /identity/v3/projects/4a54457d6f9a41d4bfa1f4b3759b09a9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:40.245227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e631ce-1542-4ffe-b898-2c0bba5d69e8 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.245513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e631ce-1542-4ffe-b898-2c0bba5d69e8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.245783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e631ce-1542-4ffe-b898-2c0bba5d69e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.270611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-94e631ce-1542-4ffe-b898-2c0bba5d69e8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:40.273016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-94e631ce-1542-4ffe-b898-2c0bba5d69e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.293737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:40.295301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.368627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 986/1989] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:40.378073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9ba6-d3fc-4705-b3a3-1e596468d62d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.378449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9ba6-d3fc-4705-b3a3-1e596468d62d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.378967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9ba6-d3fc-4705-b3a3-1e596468d62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.406673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:40.425159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08678990-f335-4dd1-ae26-b77a8d198702 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:40.438487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:40.438933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-285c9ba6-d3fc-4705-b3a3-1e596468d62d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:40.440798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.441191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.441615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] PATH_INFO: `/v3/projects/646a0f6383bc4392807aa97c71d8638c` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.442110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-285c9ba6-d3fc-4705-b3a3-1e596468d62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.445033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646a0f6383bc4392807aa97c71d8638c)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:40.447216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:40.454226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 987/1990] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:40.460483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9ba6-d3fc-4705-b3a3-1e596468d62d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:40.483611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9ba6-d3fc-4705-b3a3-1e596468d62d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:40.489192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:40.492542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.493053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.493558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.494437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:40.494919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:40.496847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:40.512753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646a0f6383bc4392807aa97c71d8638c', 'request_id': 'req-d2ae5938-1d2e-43e8-8923-6413196894b4'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.514698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 988/1991] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:40.519289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:40.523557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646a0f6383bc4392807aa97c71d8638c', 'request_id': 'req-d2ae5938-1d2e-43e8-8923-6413196894b4'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.533035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646a0f6383bc4392807aa97c71d8638c', 'request_id': 'req-d2ae5938-1d2e-43e8-8923-6413196894b4'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.539146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b564b-f856-43b8-b031-a37b2340e99e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:40.563028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] The token cache is being invalidate because project 646a0f6383bc4392807aa97c71d8638c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:40.567738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.568753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d2ae5938-1d2e-43e8-8923-6413196894b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.572826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1004/1992] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:40 2022] DELETE /identity/v3/projects/646a0f6383bc4392807aa97c71d8638c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:40.577842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.578197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.579140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.627698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:40.630741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.639009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:40.660947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:40.662988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.663271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.663539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.664235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:40.669496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:40.727814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 989/1993] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:40 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:40.728736 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:40.730329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.786615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1005/1994] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:40.787535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:40.789441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.809341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14585701-5b02-400b-887b-9b8cec4455e9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:40.817522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed6f1d-adf1-46bf-a890-8494bb34ac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:40.817522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed6f1d-adf1-46bf-a890-8494bb34ac51 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.817522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed6f1d-adf1-46bf-a890-8494bb34ac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.817522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed6f1d-adf1-46bf-a890-8494bb34ac51 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.817993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed6f1d-adf1-46bf-a890-8494bb34ac51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:40.819210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed6f1d-adf1-46bf-a890-8494bb34ac51 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:40.844374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1006/1995] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:40 2022] POST /identity/v3/users => generated 292 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:40.845732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.845989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790bd2b-db2d-48fe-b1fb-0e0bf5c8d012 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:40.846109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.846281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.855338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:40.856028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:40.856291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:40.856588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] PATH_INFO: `/v3/projects/401bfe45dca84dbab522f6a25d8daf96` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:40.858583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=401bfe45dca84dbab522f6a25d8daf96)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:40.860527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:40.885873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:40.887094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.895346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1007/1996] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:40.899992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:40.901478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.916823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c071b8-3b76-44e0-ba5f-e514c3468388 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:40.932461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401bfe45dca84dbab522f6a25d8daf96', 'request_id': 'req-3a49ad27-853f-45a9-a77a-d52d1dd009ce'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.938375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401bfe45dca84dbab522f6a25d8daf96', 'request_id': 'req-3a49ad27-853f-45a9-a77a-d52d1dd009ce'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.944939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401bfe45dca84dbab522f6a25d8daf96', 'request_id': 'req-3a49ad27-853f-45a9-a77a-d52d1dd009ce'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.959373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] The token cache is being invalidate because project 401bfe45dca84dbab522f6a25d8daf96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:40.959735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.961121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:40.962814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 990/1997] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:40 2022] DELETE /identity/v3/projects/401bfe45dca84dbab522f6a25d8daf96 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:40.963737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:40.966845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:40.981928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:40.998051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:40.999265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.001485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.002054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] PATH_INFO: `/v3/projects/9c4266e041fd4ac7b33d0297c209d984` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.004412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4266e041fd4ac7b33d0297c209d984)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.007123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.045356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49ad27-853f-45a9-a77a-d52d1dd009ce admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.053024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.054502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.055207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.056859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.057660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.060130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.065895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4266e041fd4ac7b33d0297c209d984', 'request_id': 'req-e40c6d05-0390-421d-9db4-85c9d9fa918e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.071825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4266e041fd4ac7b33d0297c209d984', 'request_id': 'req-e40c6d05-0390-421d-9db4-85c9d9fa918e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.072649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 991/1998] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:41.073655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.077397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.086130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:41.089010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4266e041fd4ac7b33d0297c209d984', 'request_id': 'req-e40c6d05-0390-421d-9db4-85c9d9fa918e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.092449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec0e51-2c1d-45f4-b250-e40d47453c34 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.109396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] The token cache is being invalidate because project 9c4266e041fd4ac7b33d0297c209d984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:41.109859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.110593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.115666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1008/1999] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:40 2022] DELETE /identity/v3/projects/9c4266e041fd4ac7b33d0297c209d984 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:41.115666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.115666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.193938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.194571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.194857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.195599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.195843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:41.196152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.197868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.219871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 992/2000] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:41.219871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.219871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.220130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c6d05-0390-421d-9db4-85c9d9fa918e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.228846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36855f7-d0ad-42ef-b586-33fd47522e85 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.234232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b5191-0ece-4d61-a6b0-173d3e461eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.235641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b5191-0ece-4d61-a6b0-173d3e461eea admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.235641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b5191-0ece-4d61-a6b0-173d3e461eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.236339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b5191-0ece-4d61-a6b0-173d3e461eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.237066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b5191-0ece-4d61-a6b0-173d3e461eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.238794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.240176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b5191-0ece-4d61-a6b0-173d3e461eea admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.240303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.240708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.241124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] PATH_INFO: `/v3/users/7458cf1683fe45c9874d31a7905513ea` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.242738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7458cf1683fe45c9874d31a7905513ea)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.245705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.254211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 993/2001] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:41.255637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.255902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.256177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.278428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:41.279714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.299632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 994/2002] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:41.299632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.299632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.312325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31caa9-562a-411c-8f42-527ae9aa9251 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.317179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db217d8a-d37c-47f9-8313-36d66b978521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.317901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217d8a-d37c-47f9-8313-36d66b978521 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.318420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217d8a-d37c-47f9-8313-36d66b978521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.318634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217d8a-d37c-47f9-8313-36d66b978521 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.319492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db217d8a-d37c-47f9-8313-36d66b978521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.321307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db217d8a-d37c-47f9-8313-36d66b978521 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.322170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7458cf1683fe45c9874d31a7905513ea'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.322871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7458cf1683fe45c9874d31a7905513ea'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.332359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7458cf1683fe45c9874d31a7905513ea'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.333307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7458cf1683fe45c9874d31a7905513ea'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.345132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7458cf1683fe45c9874d31a7905513ea'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:41.370849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1009/2003] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:41 2022] DELETE /identity/v3/users/7458cf1683fe45c9874d31a7905513ea => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:41.374874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.376434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.389742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:41.396417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 995/2004] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:41.400360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.400742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.404268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.433620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:41.437291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.503791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4014e9-dc88-4734-bcc2-baa161afbabc admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.527210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 996/2005] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:41.528923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.530816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.546658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b275863-6c8d-4e00-ace8-35355b162f91 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.552177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.552969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.553246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.553511 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.554137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.556081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.579031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 997/2006] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/users => generated 288 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:41.579885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.590476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac94e-e121-46c4-a97e-131555a50898 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.593071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.594082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.594220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.594395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] PATH_INFO: `/v3/projects/0856a5568fe542e6847f70851e2e2193/users/ea223d5cb73247678e1a564859191fd3/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.595818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.596432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.596839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.597112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.597766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:41.598965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.600240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.612119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0856a5568fe542e6847f70851e2e2193, user_id=ea223d5cb73247678e1a564859191fd3, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.615299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 998/2007] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:41.615431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.616402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.633918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16499a1-ae1d-4f72-aeff-f3f599f80bb2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.634593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1010/2008] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:41 2022] PUT /identity/v3/projects/0856a5568fe542e6847f70851e2e2193/users/ea223d5cb73247678e1a564859191fd3/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:41.636633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.661306 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:41.701478 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef4a07-5122-4034-81f3-e30d4a691d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.702085 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4a07-5122-4034-81f3-e30d4a691d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.702368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4a07-5122-4034-81f3-e30d4a691d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.702639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4a07-5122-4034-81f3-e30d4a691d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.703303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef4a07-5122-4034-81f3-e30d4a691d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.704961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef4a07-5122-4034-81f3-e30d4a691d07 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.716287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 999/2009] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:41.718039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f9e9a-6a69-4df7-ab91-876c18634dd5 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.720778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db74b-f3fb-40de-a14c-068bdbe45cb4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.721443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db74b-f3fb-40de-a14c-068bdbe45cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.721725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db74b-f3fb-40de-a14c-068bdbe45cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.725183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.725431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.725687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.725958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.726591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.728539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.762132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-643db74b-f3fb-40de-a14c-068bdbe45cb4 None None] MFA Rules not processed for user `ea223d5cb73247678e1a564859191fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:41.763721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-643db74b-f3fb-40de-a14c-068bdbe45cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:41.768824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1011/2010] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:41.768824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.804953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:41.855156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1000/2011] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:41.856172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db74b-f3fb-40de-a14c-068bdbe45cb4 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.857343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb8ff1-dfd3-4db1-b827-a128909d4fe4 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.862685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.863480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.864251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.864711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.865472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.867112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.871133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db74b-f3fb-40de-a14c-068bdbe45cb4 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.876702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.878473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1012/2012] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:41.879193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.879427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.880096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.880650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.881492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.884885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.892245 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a06500-d1f8-4730-b0e0-9cfb8bace62f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.900909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.901549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.901830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.902100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:41.902793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:41.905400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:41.926118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1001/2013] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/users => generated 292 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:41.927318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.940983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0d6f-98df-4330-b3c5-78d8a04a8abf admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:41.960600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1013/2014] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:41 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:41.961634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:41.963739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:41.967282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:41.968122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:41.970446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] PATH_INFO: `/v3/projects/d588ef952640489f852a4e1580794f1f/users/704306c49c4f499c80d9f3e9d2cd8d07/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.008377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d588ef952640489f852a4e1580794f1f, user_id=704306c49c4f499c80d9f3e9d2cd8d07, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.011165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.035675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1002/2015] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:41 2022] PUT /identity/v3/projects/d588ef952640489f852a4e1580794f1f/users/704306c49c4f499c80d9f3e9d2cd8d07/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:42.035675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.068305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ad41-9340-4aca-b988-ff87b60974e4 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.120970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dc7ce-1e72-43b2-ae3f-8968be30fd96 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.125855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bfcef-67b5-46ae-9c8f-9fc617e9f33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.126633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bfcef-67b5-46ae-9c8f-9fc617e9f33b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.127333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bfcef-67b5-46ae-9c8f-9fc617e9f33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.127714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bfcef-67b5-46ae-9c8f-9fc617e9f33b admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.128413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bfcef-67b5-46ae-9c8f-9fc617e9f33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.131156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bfcef-67b5-46ae-9c8f-9fc617e9f33b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.149547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:42.158879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1003/2016] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:42.161529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ea67-6b56-4996-92c9-8a81a291b764 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.161529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ea67-6b56-4996-92c9-8a81a291b764 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.161529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ea67-6b56-4996-92c9-8a81a291b764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.179565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.180694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.181240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.181787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.182448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.184543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.198430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1014/2017] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:42.198430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.198430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.199108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4a69ea67-6b56-4996-92c9-8a81a291b764 None None] MFA Rules not processed for user `704306c49c4f499c80d9f3e9d2cd8d07`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:42.200477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4a69ea67-6b56-4996-92c9-8a81a291b764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.210957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f24470-2dc8-4737-bd5f-195c5dfb5912 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.217271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.218278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.218557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.218838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] PATH_INFO: `/v3/projects/5ccfc6545eb2460fb9da04e813b3b88c` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.220018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccfc6545eb2460fb9da04e813b3b88c)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.221958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.268003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:42.276054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1004/2018] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:42.280932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69ea67-6b56-4996-92c9-8a81a291b764 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.281151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccfc6545eb2460fb9da04e813b3b88c', 'request_id': 'req-792b069c-00f1-4976-b2df-5cbceb703370'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:42.289207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccfc6545eb2460fb9da04e813b3b88c', 'request_id': 'req-792b069c-00f1-4976-b2df-5cbceb703370'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:42.297031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69ea67-6b56-4996-92c9-8a81a291b764 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.297191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccfc6545eb2460fb9da04e813b3b88c', 'request_id': 'req-792b069c-00f1-4976-b2df-5cbceb703370'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:42.312088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] The token cache is being invalidate because project 5ccfc6545eb2460fb9da04e813b3b88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:42.312412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:42.313019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:42.314842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1015/2019] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:42 2022] DELETE /identity/v3/projects/5ccfc6545eb2460fb9da04e813b3b88c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:42.317121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.319688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.365902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.366537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.366819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.367096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.367829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.369599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.386856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1005/2020] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:42.387885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.389660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.401512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b069c-00f1-4976-b2df-5cbceb703370 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.407340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799e653-fc36-4b52-a404-befe403f149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.408061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e653-fc36-4b52-a404-befe403f149a admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.408341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e653-fc36-4b52-a404-befe403f149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.408614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e653-fc36-4b52-a404-befe403f149a admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.409281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799e653-fc36-4b52-a404-befe403f149a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.411079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799e653-fc36-4b52-a404-befe403f149a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.427280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1016/2021] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:42.427280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.427280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.427280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.450664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:42.451938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.458845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad06eb-fce0-4da9-a3f0-f42c47f8df4f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.460642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1017/2022] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:42.467397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.467397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.470004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.470748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.471095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.471464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.473509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.476234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48665d-097f-459b-af88-a9c41f0cdd20 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.482073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.482806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.483152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.483489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.484252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.485958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.500340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1018/2023] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:42.500340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.508622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383caf9-55f3-43af-8c26-5c7c9d9dff6b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.514519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c6393-8ca9-49bc-bc28-75748f93fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.515220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c6393-8ca9-49bc-bc28-75748f93fd7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.515626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c6393-8ca9-49bc-bc28-75748f93fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.515904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c6393-8ca9-49bc-bc28-75748f93fd7f admin admin] PATH_INFO: `/v3/projects/c7d368b2af8a4cb3b0e254457fed4c6d/users/de2dce2e06f74244b5da166d19279015/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.527236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:42.535187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c6393-8ca9-49bc-bc28-75748f93fd7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d368b2af8a4cb3b0e254457fed4c6d, user_id=de2dce2e06f74244b5da166d19279015, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.538459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c6393-8ca9-49bc-bc28-75748f93fd7f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.553717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.555928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.564098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1019/2024] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:42 2022] PUT /identity/v3/projects/c7d368b2af8a4cb3b0e254457fed4c6d/users/de2dce2e06f74244b5da166d19279015/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:42.568015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3e52-a6c8-4251-b9a4-534e0d4de2f4 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.568323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3e52-a6c8-4251-b9a4-534e0d4de2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.568603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3e52-a6c8-4251-b9a4-534e0d4de2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.573017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1006/2025] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:42.574105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.587696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa711af3-b475-42e5-b3d0-8466d7d96a83 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.598775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-69ef3e52-a6c8-4251-b9a4-534e0d4de2f4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:42.600467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-69ef3e52-a6c8-4251-b9a4-534e0d4de2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.666962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.668561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.668852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.669431 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] PATH_INFO: `/v3/projects/3ceba900d7fb449886fc863e909a6264/users/81a50c23372c49c88a302775746e73d2/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.684451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1020/2026] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:42.685873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5d32-ad18-444a-8b74-0889b6a7dc8e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.686163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5d32-ad18-444a-8b74-0889b6a7dc8e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.686435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5d32-ad18-444a-8b74-0889b6a7dc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.694037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ceba900d7fb449886fc863e909a6264, user_id=81a50c23372c49c88a302775746e73d2, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.697566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.717138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1007/2027] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:42 2022] PUT /identity/v3/projects/3ceba900d7fb449886fc863e909a6264/users/81a50c23372c49c88a302775746e73d2/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:42.723202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-531c5d32-ad18-444a-8b74-0889b6a7dc8e None None] MFA Rules not processed for user `de2dce2e06f74244b5da166d19279015`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:42.724818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-531c5d32-ad18-444a-8b74-0889b6a7dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.727939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.737751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:42.801127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c7da6-dd24-4969-91f8-240f467b0ea6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.807191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57424ce4-4dd8-4bd3-b7e9-e2dfda82a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.808237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57424ce4-4dd8-4bd3-b7e9-e2dfda82a8bc admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.808725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57424ce4-4dd8-4bd3-b7e9-e2dfda82a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.809298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57424ce4-4dd8-4bd3-b7e9-e2dfda82a8bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.810556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57424ce4-4dd8-4bd3-b7e9-e2dfda82a8bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:42.811231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57424ce4-4dd8-4bd3-b7e9-e2dfda82a8bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.813461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57424ce4-4dd8-4bd3-b7e9-e2dfda82a8bc admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.829696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1008/2028] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:42.831327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870683-ce2c-421c-bb1c-9563564c7163 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.831884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870683-ce2c-421c-bb1c-9563564c7163 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.832379 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870683-ce2c-421c-bb1c-9563564c7163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.841757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1021/2029] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:42.844602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c5d32-ad18-444a-8b74-0889b6a7dc8e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.863259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c5d32-ad18-444a-8b74-0889b6a7dc8e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.869044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.870080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.870373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.871065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.872270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.872657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ba870683-ce2c-421c-bb1c-9563564c7163 None None] MFA Rules not processed for user `81a50c23372c49c88a302775746e73d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:42.874184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ba870683-ce2c-421c-bb1c-9563564c7163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:42.874299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:42.914875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1022/2030] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:42.915693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:42.982812 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97e218-6911-4519-9a4c-77b936b7ed9c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:42.983117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:42.988971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:42.989807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:42.990008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:42.990386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:42.991232 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:42.993425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.011271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1009/2031] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:43.011947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba870683-ce2c-421c-bb1c-9563564c7163 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.034674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba870683-ce2c-421c-bb1c-9563564c7163 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.037774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1023/2032] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:42 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:43.038640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.060298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.060298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.060298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.060538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.062002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.064754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.100791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1010/2033] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:43 2022] POST /identity/v3/users => generated 286 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:43.100791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.109808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79042548-02d8-4e08-92dc-97c18350b34b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.148027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbf4cb-22be-4baa-8ee5-55ac5c342dc3 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.153627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.154262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.154572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.154848 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.155493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.157561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.178850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:43.186382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.187146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.187446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.187730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.188372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.191371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.194967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1024/2034] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:43 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:43.195748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.219749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1011/2035] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:43 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:43.220621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.232640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9959-0fa1-4bfa-898d-cd7c0cbf0276 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.287633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e09198-9e70-41a6-bb9b-9bb6dfe0075b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.294694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.295485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.295925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.296199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.296927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.299288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.301436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.303382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.305151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.305151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.305151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.307015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.312965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1025/2036] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:43.314003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.328369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66675cff-9e75-4425-8098-3ab46a7483c9 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.334029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.334654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.334931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.335202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.335928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.337646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.337761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1012/2037] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:43 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:43.338398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.349477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1026/2038] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:43.352602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f0549-fa19-4bf3-9c91-01dcd9ff49c6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.353855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.368140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.368140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.368140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.368140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.368615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.368615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.368748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc82b-de5b-4d0f-aa56-bcb00824996e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.369066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.369966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.370255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.370527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.374500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.374500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.379204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1013/2039] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:43.380875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.386315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1027/2040] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:43.387395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.392752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e12676-32f2-4946-8219-38a50381ac46 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.397892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02fe35-3a3c-4b0a-a815-261ab29d69bc admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.400314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57584f5-5838-41d9-a564-9e63c5f9b63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.401004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57584f5-5838-41d9-a564-9e63c5f9b63e admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.401350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57584f5-5838-41d9-a564-9e63c5f9b63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.401624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57584f5-5838-41d9-a564-9e63c5f9b63e admin admin] PATH_INFO: `/v3/projects/618c58e66e9d4f36a87d8e18b484f246/users/6d708fb6ffd74d9d9d85055987305933/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.402527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.403142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.403408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.404305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] PATH_INFO: `/v3/projects/cc53e85e05b84a4790f069f13ab09743/users/a47da35ac0bf4f51a98fa3437fef1ea3/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.422050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57584f5-5838-41d9-a564-9e63c5f9b63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=618c58e66e9d4f36a87d8e18b484f246, user_id=6d708fb6ffd74d9d9d85055987305933, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.422592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc53e85e05b84a4790f069f13ab09743, user_id=a47da35ac0bf4f51a98fa3437fef1ea3, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.425028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57584f5-5838-41d9-a564-9e63c5f9b63e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.425371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.440927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1028/2041] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:43 2022] PUT /identity/v3/projects/cc53e85e05b84a4790f069f13ab09743/users/a47da35ac0bf4f51a98fa3437fef1ea3/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:43.441656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.449865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1014/2042] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:43 2022] PUT /identity/v3/projects/618c58e66e9d4f36a87d8e18b484f246/users/6d708fb6ffd74d9d9d85055987305933/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:43.452572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30ef6-cd18-4796-bf6a-062e54136ea0 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.453284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30ef6-cd18-4796-bf6a-062e54136ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.453284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30ef6-cd18-4796-bf6a-062e54136ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.479626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:43.483034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e5d30ef6-cd18-4796-bf6a-062e54136ea0 None None] MFA Rules not processed for user `a47da35ac0bf4f51a98fa3437fef1ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:43.484820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e5d30ef6-cd18-4796-bf6a-062e54136ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:43.526643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d33fdc-30a8-428c-a1b9-2050c0e7f36b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.556389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1015/2043] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:43.557285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d30ef6-cd18-4796-bf6a-062e54136ea0 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.601681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:43.612087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.613142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.613421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.613700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] PATH_INFO: `/v3/projects/3c2bb1caf27e43e1a74f4b79da50ab1d/users/8aaaafb3eaff4d49845c96f7d3955ffc/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.630760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2bb1caf27e43e1a74f4b79da50ab1d, user_id=8aaaafb3eaff4d49845c96f7d3955ffc, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.633657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.650612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1029/2044] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:43 2022] PUT /identity/v3/projects/3c2bb1caf27e43e1a74f4b79da50ab1d/users/8aaaafb3eaff4d49845c96f7d3955ffc/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:43.651789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.653648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:43.655212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d30ef6-cd18-4796-bf6a-062e54136ea0 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.722955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.723667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.724349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.724349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.724958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.726705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.737563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1016/2045] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:43.738569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.746675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ceaf-4b10-49fd-92ab-1f88aa83b7aa admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.749272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855bb30-eb2d-48a7-bac7-7ac0898d6824 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.750664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c0c1-168b-4e49-b19b-a67d589754ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.751412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c0c1-168b-4e49-b19b-a67d589754ca service glance] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.751814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c0c1-168b-4e49-b19b-a67d589754ca service glance] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.752152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c0c1-168b-4e49-b19b-a67d589754ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.754291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.755338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.755794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.756207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.757092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.758830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.770714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1017/2046] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:43.774552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.783345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084415bf-2808-4751-801c-3e16fafff091 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.788911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.789859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.790156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.790433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] PATH_INFO: `/v3/projects/618c58e66e9d4f36a87d8e18b484f246/users/6d708fb6ffd74d9d9d85055987305933/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.793218 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] RBAC: Authorizing `identity:create_grant(project_id=618c58e66e9d4f36a87d8e18b484f246, user_id=6d708fb6ffd74d9d9d85055987305933, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.795891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.808627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1018/2047] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:43 2022] PUT /identity/v3/projects/618c58e66e9d4f36a87d8e18b484f246/users/6d708fb6ffd74d9d9d85055987305933/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:43.809655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:43.830120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511c0c1-168b-4e49-b19b-a67d589754ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.831605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511c0c1-168b-4e49-b19b-a67d589754ca service glance] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.841789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1030/2048] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:43.843592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91674eb8-1b76-46c9-b9bd-dc8b318373cf None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.843949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91674eb8-1b76-46c9-b9bd-dc8b318373cf None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.844310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91674eb8-1b76-46c9-b9bd-dc8b318373cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.847851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:43.876561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-91674eb8-1b76-46c9-b9bd-dc8b318373cf None None] MFA Rules not processed for user `6d708fb6ffd74d9d9d85055987305933`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:43.878251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-91674eb8-1b76-46c9-b9bd-dc8b318373cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:43.923283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2790568-852a-429c-b2ac-50f4bc9fb34a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:43.932139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7f50f-fd68-4c64-a30c-dc2c7e572fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:43.933050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f50f-fd68-4c64-a30c-dc2c7e572fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.933490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f50f-fd68-4c64-a30c-dc2c7e572fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.933917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f50f-fd68-4c64-a30c-dc2c7e572fe3 admin admin] PATH_INFO: `/v3/projects/3c2bb1caf27e43e1a74f4b79da50ab1d/users/8aaaafb3eaff4d49845c96f7d3955ffc/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.937725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7f50f-fd68-4c64-a30c-dc2c7e572fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2bb1caf27e43e1a74f4b79da50ab1d, user_id=8aaaafb3eaff4d49845c96f7d3955ffc, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:43.941092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7f50f-fd68-4c64-a30c-dc2c7e572fe3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:43.961718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:43.967966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1019/2049] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:43 2022] PUT /identity/v3/projects/3c2bb1caf27e43e1a74f4b79da50ab1d/users/8aaaafb3eaff4d49845c96f7d3955ffc/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:43.978858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b13b9b-c9a6-42b7-adcc-aef95ca4cfb2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:43.979193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b13b9b-c9a6-42b7-adcc-aef95ca4cfb2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:43.979436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b13b9b-c9a6-42b7-adcc-aef95ca4cfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:43.979931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1031/2050] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:43.991730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91674eb8-1b76-46c9-b9bd-dc8b318373cf None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:44.018489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a5b13b9b-c9a6-42b7-adcc-aef95ca4cfb2 None None] MFA Rules not processed for user `8aaaafb3eaff4d49845c96f7d3955ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:44.020432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a5b13b9b-c9a6-42b7-adcc-aef95ca4cfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:44.057156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91674eb8-1b76-46c9-b9bd-dc8b318373cf None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:44.119271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:44.120096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:44.120249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1020/2051] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:44.120368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:44.120639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:44.205436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:44.207184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:44.220829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1032/2052] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:39:44.298651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b13b9b-c9a6-42b7-adcc-aef95ca4cfb2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:44.309438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b13b9b-c9a6-42b7-adcc-aef95ca4cfb2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:44.313377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:44.314095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:44.314381 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:44.314663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:44.329916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:44.331486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:44.352903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1021/2053] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:44.522819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:44.537444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1765d42-40af-4bc9-b941-566cbc904a92 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:44.541527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:44.542276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:44.542668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:44.542951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:44.624296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:44.625876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:44.631956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:44.637730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1033/2054] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:45.064061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:45.066607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:45.085539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb262b9-a107-4aeb-b3d2-464550e4b3b5 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:45.089345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedff7d-0d4f-45d6-98f7-67bf16b79d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:45.089974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedff7d-0d4f-45d6-98f7-67bf16b79d0a service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:45.090223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedff7d-0d4f-45d6-98f7-67bf16b79d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:45.090490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedff7d-0d4f-45d6-98f7-67bf16b79d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:45.092138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3aedff7d-0d4f-45d6-98f7-67bf16b79d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:45.152415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedff7d-0d4f-45d6-98f7-67bf16b79d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:45.153856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedff7d-0d4f-45d6-98f7-67bf16b79d0a service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:45.163278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1022/2055] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:53.706437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:53.775883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbff443-fa42-48a8-85fb-1840beadeb56 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:53.781144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:53.782095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:53.782339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:53.782688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] PATH_INFO: `/v3/users/de2dce2e06f74244b5da166d19279015` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:53.783874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2dce2e06f74244b5da166d19279015)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:53.785881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:53.838703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2dce2e06f74244b5da166d19279015'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:53.839490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2dce2e06f74244b5da166d19279015'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:53.849624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2dce2e06f74244b5da166d19279015'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:53.850462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2dce2e06f74244b5da166d19279015'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:53.859977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2dce2e06f74244b5da166d19279015'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:53.878142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1034/2056] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:53 2022] DELETE /identity/v3/users/de2dce2e06f74244b5da166d19279015 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:53.882806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:53.883397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:53.883512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:53.907842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:53.909066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:53.931088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:53.976381 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1023/2057] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:53.987718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:53.989621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:54.064164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1947a-29a2-49e6-8f35-d4af1976343f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:54.069001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:54.069638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:54.070436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:54.070436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:54.081298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:54.082934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:54.100869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1035/2058] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:54.485237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:54.488236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:54.501943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aee27-c81b-405c-b8a8-b71c4be4736d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:54.508271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:54.508988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:54.509259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:54.509536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] PATH_INFO: `/v3/users/640b03dc796542249c0c6fa7c86e7fe9` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:54.511192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640b03dc796542249c0c6fa7c86e7fe9)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:54.514198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:54.515322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:54.516865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:54.541224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c52c-b86e-4ab1-b605-39e3eaa1332c service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:54.549152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:54.549805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:54.550084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:54.550356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] PATH_INFO: `/v3/projects/c7d368b2af8a4cb3b0e254457fed4c6d` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:54.551613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d368b2af8a4cb3b0e254457fed4c6d)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:54.553788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:54.574558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:54.577208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640b03dc796542249c0c6fa7c86e7fe9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.578367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640b03dc796542249c0c6fa7c86e7fe9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.579405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640b03dc796542249c0c6fa7c86e7fe9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.592092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640b03dc796542249c0c6fa7c86e7fe9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.600011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f47e597b-3e35-44f0-ada3-3fc0a1430d74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640b03dc796542249c0c6fa7c86e7fe9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.634900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d368b2af8a4cb3b0e254457fed4c6d', 'request_id': 'req-5c6009bc-6073-4922-9fb7-50393536006d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.640348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1024/2059] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:54 2022] DELETE /identity/v3/users/640b03dc796542249c0c6fa7c86e7fe9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:54.645775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:54.645902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d368b2af8a4cb3b0e254457fed4c6d', 'request_id': 'req-5c6009bc-6073-4922-9fb7-50393536006d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.646823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:54.647420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:54.656456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d368b2af8a4cb3b0e254457fed4c6d', 'request_id': 'req-5c6009bc-6073-4922-9fb7-50393536006d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.672263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:54.673856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:54.675816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] The token cache is being invalidate because project c7d368b2af8a4cb3b0e254457fed4c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:54.676130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.676793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.681527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1036/2060] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:54 2022] DELETE /identity/v3/projects/c7d368b2af8a4cb3b0e254457fed4c6d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:54.682865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:54.685527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:54.752997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1025/2061] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:54.762939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:54.764459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:54.773520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6009bc-6073-4922-9fb7-50393536006d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:54.779365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:54.780044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:54.780327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:54.780605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] PATH_INFO: `/v3/users/6d708fb6ffd74d9d9d85055987305933` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:54.781753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d708fb6ffd74d9d9d85055987305933)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:54.784012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:54.809130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:54.826553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2a74-e89f-4631-b1c0-42c634764dce None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:54.831601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:54.832236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:54.832517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:54.832810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:54.838689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d708fb6ffd74d9d9d85055987305933'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.838689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d708fb6ffd74d9d9d85055987305933'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.843350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:54.844826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:54.847236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d708fb6ffd74d9d9d85055987305933'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.848049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d708fb6ffd74d9d9d85055987305933'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.857278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d708fb6ffd74d9d9d85055987305933'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:54.859903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1026/2062] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:54.880063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1037/2063] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:54 2022] DELETE /identity/v3/users/6d708fb6ffd74d9d9d85055987305933 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:55.237595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.244061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.245967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:55.249594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5359cc1-1502-40fe-a619-99bff744a5aa service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.284110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:55.353575 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026dea6-e23e-438b-b3f7-216cf6a26199 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.356401 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.357150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.357505 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.357848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] PATH_INFO: `/v3/projects/618c58e66e9d4f36a87d8e18b484f246` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.359333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618c58e66e9d4f36a87d8e18b484f246)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.361651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.361878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.362532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.362807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.363075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] PATH_INFO: `/v3/projects/68eebfe8c9654799a66fb3f9f09578b1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.364301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68eebfe8c9654799a66fb3f9f09578b1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.366214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.423409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618c58e66e9d4f36a87d8e18b484f246', 'request_id': 'req-dc196101-d178-4392-8489-afcb46c17845'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.425690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68eebfe8c9654799a66fb3f9f09578b1', 'request_id': 'req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.429944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618c58e66e9d4f36a87d8e18b484f246', 'request_id': 'req-dc196101-d178-4392-8489-afcb46c17845'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.431737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68eebfe8c9654799a66fb3f9f09578b1', 'request_id': 'req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.438594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68eebfe8c9654799a66fb3f9f09578b1', 'request_id': 'req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.438754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618c58e66e9d4f36a87d8e18b484f246', 'request_id': 'req-dc196101-d178-4392-8489-afcb46c17845'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.454019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] The token cache is being invalidate because project 68eebfe8c9654799a66fb3f9f09578b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:55.454307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.454938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.456495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1038/2064] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:55 2022] DELETE /identity/v3/projects/68eebfe8c9654799a66fb3f9f09578b1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:55.459484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] The token cache is being invalidate because project 618c58e66e9d4f36a87d8e18b484f246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:55.460055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.460409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.460791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dc196101-d178-4392-8489-afcb46c17845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.461996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:55.466748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1027/2065] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:55 2022] DELETE /identity/v3/projects/618c58e66e9d4f36a87d8e18b484f246 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:55.468902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d731ac7-c937-4520-82b4-0d335cf1d35f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.469032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d731ac7-c937-4520-82b4-0d335cf1d35f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.469282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d731ac7-c937-4520-82b4-0d335cf1d35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.490753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-7d731ac7-c937-4520-82b4-0d335cf1d35f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:55.492406 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7d731ac7-c937-4520-82b4-0d335cf1d35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:55.532470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e21aed-2ea6-43c1-8a22-6b60dad4af50 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.537018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.537674 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.537948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.538218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] PATH_INFO: `/v3/users/dca0d7ea822a475aa7747b61d51fa23b` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.539345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca0d7ea822a475aa7747b61d51fa23b)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.541372 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.555366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1028/2066] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:55.563371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d94cff-bed1-494c-b42d-a02bbb230696 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.563836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d94cff-bed1-494c-b42d-a02bbb230696 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.564108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d94cff-bed1-494c-b42d-a02bbb230696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.585588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-42d94cff-bed1-494c-b42d-a02bbb230696 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:55.586899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-42d94cff-bed1-494c-b42d-a02bbb230696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:55.591340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca0d7ea822a475aa7747b61d51fa23b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.592028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca0d7ea822a475aa7747b61d51fa23b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.596628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1029/2067] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:55.600297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca0d7ea822a475aa7747b61d51fa23b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.601012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca0d7ea822a475aa7747b61d51fa23b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.601514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d94cff-bed1-494c-b42d-a02bbb230696 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.602915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:55.610593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca0d7ea822a475aa7747b61d51fa23b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:55.611666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d94cff-bed1-494c-b42d-a02bbb230696 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.617338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.618046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.618321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.618583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.619404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:55.619732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.621342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.622902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:55.630593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1039/2068] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:55 2022] DELETE /identity/v3/users/dca0d7ea822a475aa7747b61d51fa23b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:55.635627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1030/2069] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:55.639752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.706956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1400285-2b82-4e5e-9cf6-ad94208fc0f3 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.713481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.714106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.714420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.714679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.715454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.717470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.751746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1040/2070] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:55 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:55.754570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.765624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744b88f-bd30-4346-bdd4-b0f3d2cd7afe admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.825568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.826190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.826468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.826744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.827373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.829325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.852001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1031/2071] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:55 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:55.856312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.868389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb4572-7609-4f90-8bd7-b087e08d38ee admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.873713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.874324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.874598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.874864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.875513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.877170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.888353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1041/2072] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:55.892739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.911549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d81b9-c75c-4ed7-b7b8-3887f72780a5 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.917447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.918503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.918976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.919465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.920949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.924464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.927830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.930240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:55.943927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7871f25-32ea-43d0-893c-b99221d5118c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.945069 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1032/2073] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:55.949147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:55.954672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.955717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.956152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.956582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] PATH_INFO: `/v3/projects/e2651c5c5cdf49a28fb89030ca05e2fd` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.959268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2651c5c5cdf49a28fb89030ca05e2fd)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:55.963002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:55.965783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13f544-5fc5-44b1-a874-2347ef5fad54 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:55.973157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db130a8-43f7-4a45-b58a-057cd10cab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:55.974191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db130a8-43f7-4a45-b58a-057cd10cab4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:55.974516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db130a8-43f7-4a45-b58a-057cd10cab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:55.974939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db130a8-43f7-4a45-b58a-057cd10cab4e admin admin] PATH_INFO: `/v3/projects/5fd5715022ad4aa29b54cad0bb33b88a/users/e5c3713b01c04a8794fbcbd5969f2279/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:55.998286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db130a8-43f7-4a45-b58a-057cd10cab4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd5715022ad4aa29b54cad0bb33b88a, user_id=e5c3713b01c04a8794fbcbd5969f2279, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.001149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db130a8-43f7-4a45-b58a-057cd10cab4e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.016501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1033/2074] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:55 2022] PUT /identity/v3/projects/5fd5715022ad4aa29b54cad0bb33b88a/users/e5c3713b01c04a8794fbcbd5969f2279/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:56.032359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2651c5c5cdf49a28fb89030ca05e2fd', 'request_id': 'req-d70814ab-7d57-40eb-bbde-1f3615f3ae55'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:56.038081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9681039-dcda-44db-be12-264c3577b100 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.038654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9681039-dcda-44db-be12-264c3577b100 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.039052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9681039-dcda-44db-be12-264c3577b100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.043172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2651c5c5cdf49a28fb89030ca05e2fd', 'request_id': 'req-d70814ab-7d57-40eb-bbde-1f3615f3ae55'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:56.049764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2651c5c5cdf49a28fb89030ca05e2fd', 'request_id': 'req-d70814ab-7d57-40eb-bbde-1f3615f3ae55'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:56.065497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] The token cache is being invalidate because project e2651c5c5cdf49a28fb89030ca05e2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:56.065787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:56.066433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-d70814ab-7d57-40eb-bbde-1f3615f3ae55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:56.067987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1042/2075] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:55 2022] DELETE /identity/v3/projects/e2651c5c5cdf49a28fb89030ca05e2fd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:56.072855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d296bb-d4f1-484b-9976-76c7408e610b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.073134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d296bb-d4f1-484b-9976-76c7408e610b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.073403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d296bb-d4f1-484b-9976-76c7408e610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.074390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f9681039-dcda-44db-be12-264c3577b100 None None] MFA Rules not processed for user `e5c3713b01c04a8794fbcbd5969f2279`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:56.075658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f9681039-dcda-44db-be12-264c3577b100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:56.094095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c5d296bb-d4f1-484b-9976-76c7408e610b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:56.095303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c5d296bb-d4f1-484b-9976-76c7408e610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:56.121387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:56.143173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1034/2076] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:56.147837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9681039-dcda-44db-be12-264c3577b100 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.149149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f9681039-dcda-44db-be12-264c3577b100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:56.157339 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1043/2077] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:56.162987 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9681039-dcda-44db-be12-264c3577b100 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.165476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3a7f9-af8d-4446-8a37-2c0c3e6d1f4c None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.165476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3a7f9-af8d-4446-8a37-2c0c3e6d1f4c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.165887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3a7f9-af8d-4446-8a37-2c0c3e6d1f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.168642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.169254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.169663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.169776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.170416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.172354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.185655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-98b3a7f9-af8d-4446-8a37-2c0c3e6d1f4c None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:56.186968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-98b3a7f9-af8d-4446-8a37-2c0c3e6d1f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:56.195371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1044/2078] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:56.199654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3a7f9-af8d-4446-8a37-2c0c3e6d1f4c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.203867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1035/2079] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:56.208758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3a7f9-af8d-4446-8a37-2c0c3e6d1f4c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.216025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.234669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:56.271287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.271887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.272170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.272444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.273127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:56.273431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.275156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.278928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dea34-01e0-4fbb-a1f4-ce3be7b336e3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.283988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.284604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.284879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.285148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.285778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.286603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1045/2080] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:39:56.287707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.291216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.301576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef46a-6922-4aad-9438-de0ceddce222 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.306617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.307221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.307500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.307812 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.308437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.313146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1036/2081] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:56.313297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.314116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.324501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc57c-64eb-4642-acf4-61f72a3498a6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.329859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.330516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.330790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.331076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.331767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.333347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.344447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1046/2082] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:56.344630 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1037/2083] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:56.345808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.348408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.359119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7b792-74a4-45ca-9f4c-cdbb6a3831d0 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.417155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f132c-19a6-4a67-80f1-b3383ba1c758 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.422186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.422954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.423333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.423757 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.424494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.426581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.434706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.435617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.435907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.436746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] PATH_INFO: `/v3/projects/b1dafba5e88644db899a8d5ab9584c1e/users/2993274497e144f7a578020b7bc4b5ee/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.453145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1dafba5e88644db899a8d5ab9584c1e, user_id=2993274497e144f7a578020b7bc4b5ee, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.456485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1047/2084] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:56.456649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.457511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.469621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e2fe-ada1-4a4c-94e2-0c3a97a2d8e6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.472978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1038/2085] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:56 2022] PUT /identity/v3/projects/b1dafba5e88644db899a8d5ab9584c1e/users/2993274497e144f7a578020b7bc4b5ee/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:56.477680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.497229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:56.537782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.538429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.538707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.538983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.539806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.541675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.543235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b6947-353b-4378-aa28-f76cf8f9daa7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.547898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.548509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.548785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.549053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.549699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.551280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.553704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1048/2086] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:56.557692 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.565517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1039/2087] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:56.566835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.568401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a71a0c-a555-41db-88c2-089a8a847529 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.573289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.573897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.574709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.574709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.575063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.576707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.578203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6a8dc-7ca0-4ac5-9bb5-16d3cab3d60b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.584825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be52a37-ba09-4727-9735-73f6a4043a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.585469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be52a37-ba09-4727-9735-73f6a4043a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.585744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be52a37-ba09-4727-9735-73f6a4043a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.586013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be52a37-ba09-4727-9735-73f6a4043a9c admin admin] PATH_INFO: `/v3/projects/b1dafba5e88644db899a8d5ab9584c1e/users/2993274497e144f7a578020b7bc4b5ee/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.587889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1049/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:56.588603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be52a37-ba09-4727-9735-73f6a4043a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1dafba5e88644db899a8d5ab9584c1e, user_id=2993274497e144f7a578020b7bc4b5ee, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.591270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be52a37-ba09-4727-9735-73f6a4043a9c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.591904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.598514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:56.611617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1040/2089] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:56 2022] PUT /identity/v3/projects/b1dafba5e88644db899a8d5ab9584c1e/users/2993274497e144f7a578020b7bc4b5ee/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:56.611797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d16c7a-4bcb-40f6-b2c6-f6c503b693a8 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.614661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.615013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.615341 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.645119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] MFA Rules not processed for user `2993274497e144f7a578020b7bc4b5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:56.646337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:56.673483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda50cb0-dcf5-46c7-b32f-5b9d4cde024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.674718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda50cb0-dcf5-46c7-b32f-5b9d4cde024d admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.674718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda50cb0-dcf5-46c7-b32f-5b9d4cde024d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.675045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda50cb0-dcf5-46c7-b32f-5b9d4cde024d admin admin] PATH_INFO: `/v3/projects/a7bd5ddca012429a8386f5b2b22dcffa/users/115bc5b16a944e49b7b922a04f3051ac/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.690806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda50cb0-dcf5-46c7-b32f-5b9d4cde024d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bd5ddca012429a8386f5b2b22dcffa, user_id=115bc5b16a944e49b7b922a04f3051ac, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.693705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda50cb0-dcf5-46c7-b32f-5b9d4cde024d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.707633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1050/2090] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:56 2022] PUT /identity/v3/projects/a7bd5ddca012429a8386f5b2b22dcffa/users/115bc5b16a944e49b7b922a04f3051ac/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:56.718410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1041/2091] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:56.719604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d98c80-1c39-4bd9-bace-b56e7b57dd7b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.719902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d98c80-1c39-4bd9-bace-b56e7b57dd7b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.720177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d98c80-1c39-4bd9-bace-b56e7b57dd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.730021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.731483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:56.747601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-93d98c80-1c39-4bd9-bace-b56e7b57dd7b None None] MFA Rules not processed for user `115bc5b16a944e49b7b922a04f3051ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:56.748810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-93d98c80-1c39-4bd9-bace-b56e7b57dd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:56.795619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546e454-42fb-4a20-8d41-9eabd5e0bf5d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.822459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1051/2092] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:56.826982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d98c80-1c39-4bd9-bace-b56e7b57dd7b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.848829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.849450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.849732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.850001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.855424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:56.893755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d98c80-1c39-4bd9-bace-b56e7b57dd7b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:56.901303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:56.902016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:56.902396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:56.902764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:56.903535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.905786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.906497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:56.917286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:56.919137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:56.929506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1042/2093] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:39:56.936015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1052/2094] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:56.940167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:56.950986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2a9d4-e39b-451c-8ddd-ac8ff8713215 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.022099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.022681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.022794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.022980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.023245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.023907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.025873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.048561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1043/2095] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:56 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:39:57.052480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.062208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173be70-3e7d-4a6c-84e1-8df195361468 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.067124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.067792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.068073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.068338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.068977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.070567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.081220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1044/2096] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:39:57.083327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80101a07-d3ae-4fd1-8b16-eaeab60dea37 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.085149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.087872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.088595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.088931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.089269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.095412 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7873a-8921-433a-a81c-60d10d375218 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.099893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.100525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.100802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.101084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] PATH_INFO: `/v3/projects/8142f1ab8a3549c79774b3579efa9b63/users/bfa242c07feb4dad9eae760737071018/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.116221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8142f1ab8a3549c79774b3579efa9b63, user_id=bfa242c07feb4dad9eae760737071018, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.119202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.135128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1045/2097] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:57 2022] PUT /identity/v3/projects/8142f1ab8a3549c79774b3579efa9b63/users/bfa242c07feb4dad9eae760737071018/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:57.139006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.148978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.151213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.160369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1053/2098] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:39:57.202191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e5277-8d9e-4f1f-b576-3cb52b39ad8c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.207147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b7ad1-671f-4302-9297-61221dd8b4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.207786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b7ad1-671f-4302-9297-61221dd8b4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.208062 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b7ad1-671f-4302-9297-61221dd8b4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.208352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b7ad1-671f-4302-9297-61221dd8b4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.209017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b7ad1-671f-4302-9297-61221dd8b4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.210629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b7ad1-671f-4302-9297-61221dd8b4f5 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.223554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1046/2099] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:57.228707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.240959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29dba5-e539-4ea8-88df-51d62f45826a service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.247782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.248458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.248737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.249015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] PATH_INFO: `/v3/projects/8142f1ab8a3549c79774b3579efa9b63/users/bfa242c07feb4dad9eae760737071018/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.251431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8142f1ab8a3549c79774b3579efa9b63, user_id=bfa242c07feb4dad9eae760737071018, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.254806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.271299 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1054/2100] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:57 2022] PUT /identity/v3/projects/8142f1ab8a3549c79774b3579efa9b63/users/bfa242c07feb4dad9eae760737071018/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:39:57.278027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79adb1-aba3-49cf-aa1a-f5822757ee26 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.278312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79adb1-aba3-49cf-aa1a-f5822757ee26 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.278582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79adb1-aba3-49cf-aa1a-f5822757ee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.306216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4f79adb1-aba3-49cf-aa1a-f5822757ee26 None None] MFA Rules not processed for user `bfa242c07feb4dad9eae760737071018`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:57.307601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4f79adb1-aba3-49cf-aa1a-f5822757ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:57.338193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:57.380786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1047/2101] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:57 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:57.392358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.394043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:57.469641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb077561-c9cd-493d-8415-8cb10acda98c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.479822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bffe71-8f11-41b0-9e04-de6efec1781f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.479822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bffe71-8f11-41b0-9e04-de6efec1781f service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.479822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bffe71-8f11-41b0-9e04-de6efec1781f service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.479822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bffe71-8f11-41b0-9e04-de6efec1781f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.491521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:57.496404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bffe71-8f11-41b0-9e04-de6efec1781f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.498143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bffe71-8f11-41b0-9e04-de6efec1781f service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.517158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1055/2102] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:57 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:39:57.673939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79adb1-aba3-49cf-aa1a-f5822757ee26 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.741118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79adb1-aba3-49cf-aa1a-f5822757ee26 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:57.746787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:57.747481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.747788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.748066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] PATH_INFO: `/v3/users/115bc5b16a944e49b7b922a04f3051ac` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.749226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=115bc5b16a944e49b7b922a04f3051ac)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:57.751226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:57.799421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115bc5b16a944e49b7b922a04f3051ac'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:57.800091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115bc5b16a944e49b7b922a04f3051ac'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:57.800692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115bc5b16a944e49b7b922a04f3051ac'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:57.809479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115bc5b16a944e49b7b922a04f3051ac'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:57.819259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115bc5b16a944e49b7b922a04f3051ac'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:57.845150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1048/2103] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:57 2022] DELETE /identity/v3/users/115bc5b16a944e49b7b922a04f3051ac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:57.846462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:57.846773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:57.847041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:57.867310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:57.869399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:57.936917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1056/2104] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:57.946797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:57.948233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:58.006971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:58.013714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f8a9b-b242-4df5-af75-a70b18f8a45b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:58.019136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:58.019795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:58.020071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:58.020339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:58.029962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:58.031440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:58.041465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1049/2105] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:39:58.338156 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:58.339713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:58.357435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da49833-c967-4490-9447-f06a74e50b59 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:58.362998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:58.363630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:58.364019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:58.364347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] PATH_INFO: `/v3/projects/a7bd5ddca012429a8386f5b2b22dcffa` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:58.365432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bd5ddca012429a8386f5b2b22dcffa)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:58.367407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:58.417368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bd5ddca012429a8386f5b2b22dcffa', 'request_id': 'req-94f828bf-aa2b-459b-baa0-989fdbb34e7f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.422734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bd5ddca012429a8386f5b2b22dcffa', 'request_id': 'req-94f828bf-aa2b-459b-baa0-989fdbb34e7f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.428879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bd5ddca012429a8386f5b2b22dcffa', 'request_id': 'req-94f828bf-aa2b-459b-baa0-989fdbb34e7f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.444353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] The token cache is being invalidate because project a7bd5ddca012429a8386f5b2b22dcffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:58.444635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.445234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.447292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1057/2106] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:58 2022] DELETE /identity/v3/projects/a7bd5ddca012429a8386f5b2b22dcffa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:58.453021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:58.463819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06067024-e927-4f64-9ced-8244b6e7d58e service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:58.529926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:58.530574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:58.532620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:58.532620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] PATH_INFO: `/v3/users/bfa242c07feb4dad9eae760737071018` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:58.532620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa242c07feb4dad9eae760737071018)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:58.534171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:58.604794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa242c07feb4dad9eae760737071018'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.605423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa242c07feb4dad9eae760737071018'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.606021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa242c07feb4dad9eae760737071018'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.613724 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa242c07feb4dad9eae760737071018'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.621058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-29f07b36-94be-49bf-844e-e28b20d3354d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa242c07feb4dad9eae760737071018'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:58.638149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1050/2107] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:39:58 2022] DELETE /identity/v3/users/bfa242c07feb4dad9eae760737071018 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:58.935260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:58.938453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:58.954801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f828bf-aa2b-459b-baa0-989fdbb34e7f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.027882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.028529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.028810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.029086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] PATH_INFO: `/v3/projects/8142f1ab8a3549c79774b3579efa9b63` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.030262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8142f1ab8a3549c79774b3579efa9b63)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.032261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.053791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:59.080843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8142f1ab8a3549c79774b3579efa9b63', 'request_id': 'req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:59.086150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8142f1ab8a3549c79774b3579efa9b63', 'request_id': 'req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:59.092854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8142f1ab8a3549c79774b3579efa9b63', 'request_id': 'req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:59.107802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] The token cache is being invalidate because project 8142f1ab8a3549c79774b3579efa9b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:39:59.108088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:59.108714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e43b2b86-7fd8-4ae5-a9a8-a2413ba16a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:39:59.110208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1058/2108] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:39:58 2022] DELETE /identity/v3/projects/8142f1ab8a3549c79774b3579efa9b63 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:59.116164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f32ac-7f8e-470d-b028-9ef159a72758 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.116448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f32ac-7f8e-470d-b028-9ef159a72758 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.116718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f32ac-7f8e-470d-b028-9ef159a72758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.136321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-729f32ac-7f8e-470d-b028-9ef159a72758 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:59.137551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-729f32ac-7f8e-470d-b028-9ef159a72758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:59.198045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1051/2109] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:59.205791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263325e-f103-4952-bd4e-aaaee231b443 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.206093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263325e-f103-4952-bd4e-aaaee231b443 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.206395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263325e-f103-4952-bd4e-aaaee231b443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.227758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2263325e-f103-4952-bd4e-aaaee231b443 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:59.229176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2263325e-f103-4952-bd4e-aaaee231b443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:59.237629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1059/2110] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:39:59.241885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f32ac-7f8e-470d-b028-9ef159a72758 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.251882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f32ac-7f8e-470d-b028-9ef159a72758 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.256543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.257144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.257443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.257712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.258337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:39:59.258664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.260285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.262643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:59.273704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1052/2111] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:39:59.277822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263325e-f103-4952-bd4e-aaaee231b443 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.287024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263325e-f103-4952-bd4e-aaaee231b443 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.292526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.293126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.293405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.293687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.294312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.296269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.326796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1060/2112] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:59 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:59.331036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.341757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb60f22-0365-4040-9fe2-ef6fd9825d21 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.403664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.404302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.404580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.404844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.405465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.407360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.428986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1053/2113] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:59 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:59.435141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.445892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5e8f8-3590-4425-b7d1-5eac7e4d3e35 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.451207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.451795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.452043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.452313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.452948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.454533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.466289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1061/2114] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:59.470866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.481660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b4e41-f6db-4f82-8d21-3b4531e1226a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.487484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560208b-983c-4c9f-b406-d546a99a9ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.488134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560208b-983c-4c9f-b406-d546a99a9ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.488411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560208b-983c-4c9f-b406-d546a99a9ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.488721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560208b-983c-4c9f-b406-d546a99a9ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.489349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560208b-983c-4c9f-b406-d546a99a9ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.490975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560208b-983c-4c9f-b406-d546a99a9ae4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.502181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1054/2115] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:59.506568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.517254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0e3e-ab19-41a1-b3ab-b826479f4f67 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.522439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.523112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.523370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.523669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] PATH_INFO: `/v3/projects/f503ef98fdaf4279b0013e5c1f529e57/users/9e68e99d38d0457292dcfbd240dbe041/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.538778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f503ef98fdaf4279b0013e5c1f529e57, user_id=9e68e99d38d0457292dcfbd240dbe041, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.541668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.556501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1062/2116] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:59 2022] PUT /identity/v3/projects/f503ef98fdaf4279b0013e5c1f529e57/users/9e68e99d38d0457292dcfbd240dbe041/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:59.567904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1ab49-e0a3-48ce-a2b3-d1c8405bc757 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.568196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1ab49-e0a3-48ce-a2b3-d1c8405bc757 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.568494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1ab49-e0a3-48ce-a2b3-d1c8405bc757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.605373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-97e1ab49-e0a3-48ce-a2b3-d1c8405bc757 None None] MFA Rules not processed for user `9e68e99d38d0457292dcfbd240dbe041`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:39:59.606709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-97e1ab49-e0a3-48ce-a2b3-d1c8405bc757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:39:59.674076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1055/2117] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:39:59.678818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.746929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fddee4-13be-4ff6-b3f6-d18381a9e293 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.753093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.753704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.753984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.754252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.754890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.756839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.786124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1063/2118] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:39:59 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:59.790493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1ab49-e0a3-48ce-a2b3-d1c8405bc757 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.801951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1ab49-e0a3-48ce-a2b3-d1c8405bc757 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.865248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.865893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.866176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.866447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.867087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.869173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.879168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:39:59.894986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1056/2119] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:39:59 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:39:59.901137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.912411 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6602cac-d425-41ab-9652-5a8884d95f4f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.917888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.918495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.918777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.919039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.919724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.921365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.933382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1064/2120] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:39:59.937452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:39:59.948432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4e795-6bb6-4b06-8b06-7aa46fd5e526 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:39:59.953457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:39:59.954141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:39:59.954512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:39:59.954854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] PATH_INFO: `/v3/projects/78cde154780549a8b096600953736bb3/users/0bd05d8e060944398af586951e761641/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:39:59.972589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78cde154780549a8b096600953736bb3, user_id=0bd05d8e060944398af586951e761641, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:39:59.975257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:39:59.990976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1057/2121] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:39:59 2022] PUT /identity/v3/projects/78cde154780549a8b096600953736bb3/users/0bd05d8e060944398af586951e761641/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:39:59.995351 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:00.005485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451885a-5c0b-489d-8e65-ee4b2f7101d8 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:00.018705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:00.068014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb936a6f-494b-4479-9567-16e6ac96e42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:00.068632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb936a6f-494b-4479-9567-16e6ac96e42e admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:00.068916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb936a6f-494b-4479-9567-16e6ac96e42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:00.069193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb936a6f-494b-4479-9567-16e6ac96e42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:00.069839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb936a6f-494b-4479-9567-16e6ac96e42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:00.071456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb936a6f-494b-4479-9567-16e6ac96e42e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:00.083489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1065/2122] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:00.085842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:00.096640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6b38-9735-4d53-8bbf-397af1c26178 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:00.102212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:00.102876 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:00.103164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:00.103616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] PATH_INFO: `/v3/projects/78cde154780549a8b096600953736bb3/users/0bd05d8e060944398af586951e761641/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:00.105763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=78cde154780549a8b096600953736bb3, user_id=0bd05d8e060944398af586951e761641, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:00.108425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:00.120089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1058/2123] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:00 2022] PUT /identity/v3/projects/78cde154780549a8b096600953736bb3/users/0bd05d8e060944398af586951e761641/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:00.131380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189718f5-54a8-43d2-833c-4c64ea7aff1e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:00.131938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189718f5-54a8-43d2-833c-4c64ea7aff1e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:00.132249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189718f5-54a8-43d2-833c-4c64ea7aff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:00.163920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-189718f5-54a8-43d2-833c-4c64ea7aff1e None None] MFA Rules not processed for user `0bd05d8e060944398af586951e761641`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:00.165355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-189718f5-54a8-43d2-833c-4c64ea7aff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:00.242654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1066/2124] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:00.253740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:00.255416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:00.311890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcd79a-ab61-422c-be63-7a9aabc0f33d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:00.315753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:00.316361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:00.316635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:00.316897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:00.374886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:00.377029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:00.391706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1059/2125] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:01.276006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189718f5-54a8-43d2-833c-4c64ea7aff1e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:01.286419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189718f5-54a8-43d2-833c-4c64ea7aff1e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:01.290196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12cb8f-5156-4f9b-bfee-4508674ec700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:01.290809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12cb8f-5156-4f9b-bfee-4508674ec700 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:01.291306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12cb8f-5156-4f9b-bfee-4508674ec700 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:01.292278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12cb8f-5156-4f9b-bfee-4508674ec700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:01.294098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3d12cb8f-5156-4f9b-bfee-4508674ec700 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:01.357154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12cb8f-5156-4f9b-bfee-4508674ec700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:01.358748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12cb8f-5156-4f9b-bfee-4508674ec700 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:01.367886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1067/2126] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:03.854560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:03.855997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:03.925114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083afca-e81b-4919-9347-eb2b068975c8 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:03.929671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:03.930338 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:03.930614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:03.930877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] PATH_INFO: `/v3/users/ea223d5cb73247678e1a564859191fd3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:03.932031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea223d5cb73247678e1a564859191fd3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:03.934148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:03.946763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:03.985407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea223d5cb73247678e1a564859191fd3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:03.986303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea223d5cb73247678e1a564859191fd3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:03.987099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea223d5cb73247678e1a564859191fd3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:03.999688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea223d5cb73247678e1a564859191fd3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.008086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea223d5cb73247678e1a564859191fd3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.028300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1060/2127] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:03 2022] DELETE /identity/v3/users/ea223d5cb73247678e1a564859191fd3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:04.033212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:04.034392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:04.034897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:04.056114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:04.057675 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:04.101516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:04.131493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1068/2128] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:04.141495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:04.143058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:04.224005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bed23-c795-4b71-9906-e1ee20cc3900 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:04.229054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:04.229969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:04.230246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:04.230540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:04.241170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:04.242734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:04.253301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1061/2129] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:04.684591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:04.697950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:04.705982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc7ba-91d0-4d32-884b-673db76de3f8 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:04.711388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:04.712204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:04.712498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:04.712749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] PATH_INFO: `/v3/projects/0856a5568fe542e6847f70851e2e2193` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:04.714183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0856a5568fe542e6847f70851e2e2193)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:04.716921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:04.771400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0856a5568fe542e6847f70851e2e2193', 'request_id': 'req-a9802d46-003e-43e2-ae3f-6ab52ece8213'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.778764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0856a5568fe542e6847f70851e2e2193', 'request_id': 'req-a9802d46-003e-43e2-ae3f-6ab52ece8213'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.785397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0856a5568fe542e6847f70851e2e2193', 'request_id': 'req-a9802d46-003e-43e2-ae3f-6ab52ece8213'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.805963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] The token cache is being invalidate because project 0856a5568fe542e6847f70851e2e2193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:04.806900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.807833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.809598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1069/2130] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:04 2022] DELETE /identity/v3/projects/0856a5568fe542e6847f70851e2e2193 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:04.814012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:04.826456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132160f-b025-48b2-a839-3ebce6dc1f8e service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:04.900167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:04.900882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:04.901476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:04.901822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] PATH_INFO: `/v3/users/81a50c23372c49c88a302775746e73d2` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:04.903045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81a50c23372c49c88a302775746e73d2)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:04.905212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:04.950444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:04.962102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81a50c23372c49c88a302775746e73d2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.962844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81a50c23372c49c88a302775746e73d2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.963444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81a50c23372c49c88a302775746e73d2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.972791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81a50c23372c49c88a302775746e73d2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:04.981280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81a50c23372c49c88a302775746e73d2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.001039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1062/2131] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:04 2022] DELETE /identity/v3/users/81a50c23372c49c88a302775746e73d2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:05.406853 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:05.408958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:05.423367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802d46-003e-43e2-ae3f-6ab52ece8213 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:05.489510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:05.492339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:05.511878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:05.512773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:05.513359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:05.513937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] PATH_INFO: `/v3/users/e5c3713b01c04a8794fbcbd5969f2279` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:05.515322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c3713b01c04a8794fbcbd5969f2279)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:05.517806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:05.612726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c3713b01c04a8794fbcbd5969f2279'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.613236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c3713b01c04a8794fbcbd5969f2279'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.632272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c3713b01c04a8794fbcbd5969f2279'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.633098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c3713b01c04a8794fbcbd5969f2279'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.649725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-76de96db-0afd-42ad-a452-fecaf98741cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c3713b01c04a8794fbcbd5969f2279'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.652492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:05.672808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec1a27-9907-44ca-958a-9f6f808f4758 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:05.678243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:05.679955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:05.680254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:05.680536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] PATH_INFO: `/v3/projects/3ceba900d7fb449886fc863e909a6264` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:05.681744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ceba900d7fb449886fc863e909a6264)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:05.683772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:05.687455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1070/2132] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:05 2022] DELETE /identity/v3/users/e5c3713b01c04a8794fbcbd5969f2279 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:05.689272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:05.689659 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:05.690033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:05.744615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:05.747035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:05.749518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ceba900d7fb449886fc863e909a6264', 'request_id': 'req-42a4b875-0ea4-497b-8100-6f53c6803d46'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.758483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ceba900d7fb449886fc863e909a6264', 'request_id': 'req-42a4b875-0ea4-497b-8100-6f53c6803d46'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.771882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ceba900d7fb449886fc863e909a6264', 'request_id': 'req-42a4b875-0ea4-497b-8100-6f53c6803d46'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.820297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] The token cache is being invalidate because project 3ceba900d7fb449886fc863e909a6264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:05.821172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.824288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:05.832899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1063/2133] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:05 2022] DELETE /identity/v3/projects/3ceba900d7fb449886fc863e909a6264 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:05.834288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:05.837002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:05.932239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:05.947753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1071/2134] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:05.952773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:05.957278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:05.977952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4b875-0ea4-497b-8100-6f53c6803d46 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:05.986580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:05.987778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:05.988280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:05.988763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] PATH_INFO: `/v3/users/8aaaafb3eaff4d49845c96f7d3955ffc` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:05.990161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aaaafb3eaff4d49845c96f7d3955ffc)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:05.993004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:06.038527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5b852-f054-42c8-81f6-699b4d0ea8a7 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:06.048394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:06.049081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:06.049360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:06.049852 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:06.065891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:06.067353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:06.080283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1072/2135] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:06.081843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:06.082529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aaaafb3eaff4d49845c96f7d3955ffc'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.083662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:06.104366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aaaafb3eaff4d49845c96f7d3955ffc'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.105570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aaaafb3eaff4d49845c96f7d3955ffc'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.118428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aaaafb3eaff4d49845c96f7d3955ffc'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.127426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aaaafb3eaff4d49845c96f7d3955ffc'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.174257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1064/2136] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:05 2022] DELETE /identity/v3/users/8aaaafb3eaff4d49845c96f7d3955ffc => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:06.176088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d8613-957a-4990-992c-382d6504778f service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:06.227376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:06.252459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:06.253607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:06.254124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:06.254636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:06.257159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:06.356695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:06.358431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:06.370516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1073/2137] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:06.619336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:06.621191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:06.717998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:06.719485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:06.737564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61205dc5-92fa-40a6-b9db-bbab9e0c2507 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:06.743067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:06.743964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:06.744415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:06.744878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] PATH_INFO: `/v3/projects/5fd5715022ad4aa29b54cad0bb33b88a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:06.746337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd5715022ad4aa29b54cad0bb33b88a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:06.748771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:06.814365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd5715022ad4aa29b54cad0bb33b88a', 'request_id': 'req-8bfcdffa-03a8-4fcf-892c-2436163855a7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.821788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd5715022ad4aa29b54cad0bb33b88a', 'request_id': 'req-8bfcdffa-03a8-4fcf-892c-2436163855a7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.830364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd5715022ad4aa29b54cad0bb33b88a', 'request_id': 'req-8bfcdffa-03a8-4fcf-892c-2436163855a7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.832322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:06.850769 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] The token cache is being invalidate because project 5fd5715022ad4aa29b54cad0bb33b88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:06.851109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.852133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:06.856768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1065/2138] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:06 2022] DELETE /identity/v3/projects/5fd5715022ad4aa29b54cad0bb33b88a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:06.856931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd90d1f-43f8-42bf-bbab-fea44f9517ca service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:06.858167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:06.859680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:06.932988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:06.933734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:06.934018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:06.934293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] PATH_INFO: `/v3/projects/3c2bb1caf27e43e1a74f4b79da50ab1d` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:06.935447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c2bb1caf27e43e1a74f4b79da50ab1d)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:06.937472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:06.939755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcdffa-03a8-4fcf-892c-2436163855a7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:06.945671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:06.946407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:06.946745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:06.947070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] PATH_INFO: `/v3/users/2993274497e144f7a578020b7bc4b5ee` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:06.948943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2993274497e144f7a578020b7bc4b5ee)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:06.951038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:06.986429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:06.994603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c2bb1caf27e43e1a74f4b79da50ab1d', 'request_id': 'req-51ddabd1-7300-4d4b-a30f-3ed39bf31250'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.000295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c2bb1caf27e43e1a74f4b79da50ab1d', 'request_id': 'req-51ddabd1-7300-4d4b-a30f-3ed39bf31250'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.003500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2993274497e144f7a578020b7bc4b5ee'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.004309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2993274497e144f7a578020b7bc4b5ee'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.004997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2993274497e144f7a578020b7bc4b5ee'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.006822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c2bb1caf27e43e1a74f4b79da50ab1d', 'request_id': 'req-51ddabd1-7300-4d4b-a30f-3ed39bf31250'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.016985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2993274497e144f7a578020b7bc4b5ee'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.021107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] The token cache is being invalidate because project 3c2bb1caf27e43e1a74f4b79da50ab1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:07.021399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.022017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-51ddabd1-7300-4d4b-a30f-3ed39bf31250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.023536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1074/2139] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:06 2022] DELETE /identity/v3/projects/3c2bb1caf27e43e1a74f4b79da50ab1d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:07.029231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f417b7f-4b0a-46f2-80cc-ac2d8210b881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2993274497e144f7a578020b7bc4b5ee'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.031990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ff48-277c-453e-b17d-aed32cc82d49 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.032277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ff48-277c-453e-b17d-aed32cc82d49 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.032552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ff48-277c-453e-b17d-aed32cc82d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.047925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1066/2140] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:06 2022] DELETE /identity/v3/users/2993274497e144f7a578020b7bc4b5ee => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:07.053357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-eee5ff48-277c-453e-b17d-aed32cc82d49 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:07.054555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-eee5ff48-277c-453e-b17d-aed32cc82d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.116412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1075/2141] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:07.126621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2c471-2125-4228-9c59-14e7e17bd473 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.127302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2c471-2125-4228-9c59-14e7e17bd473 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.127644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2c471-2125-4228-9c59-14e7e17bd473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.148803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6cb2c471-2125-4228-9c59-14e7e17bd473 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:07.150065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6cb2c471-2125-4228-9c59-14e7e17bd473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.159325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1067/2142] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:07.163704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5ff48-277c-453e-b17d-aed32cc82d49 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.173974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5ff48-277c-453e-b17d-aed32cc82d49 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.179243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.179901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.180176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.180440 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.181089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:07.181399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.183132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.195814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1076/2143] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:07.206315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2c471-2125-4228-9c59-14e7e17bd473 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.217026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2c471-2125-4228-9c59-14e7e17bd473 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.222091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.222702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.222975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.223238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.223903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.225789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.259959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1068/2144] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:07.261161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.271301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b7b31-daf4-4386-bcc6-d84bbed3fafe admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.329166 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.331092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.336484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.337348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.337771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.338106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.338908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.341641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.345642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928c910-7545-4eb5-bec6-109a6dd6f493 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.350878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.351795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.352072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.352334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] PATH_INFO: `/v3/projects/b1dafba5e88644db899a8d5ab9584c1e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.353454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1dafba5e88644db899a8d5ab9584c1e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.355449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.374499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1077/2145] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:07.375803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.390764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d93e1e-781b-4f80-9349-145eb56e7e39 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.396827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.397501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.397788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.398063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.398768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.400444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.409036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=3,keystone:DELETE=14,keystone:SELECT=83 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:07.413433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1078/2146] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:07.418422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.428018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1dafba5e88644db899a8d5ab9584c1e', 'request_id': 'req-987cedba-33e5-412d-81ad-7082432dcb69'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.428566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de311f-199e-4112-a811-6bfeffedd4d1 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.434404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1dafba5e88644db899a8d5ab9584c1e', 'request_id': 'req-987cedba-33e5-412d-81ad-7082432dcb69'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.442181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1dafba5e88644db899a8d5ab9584c1e', 'request_id': 'req-987cedba-33e5-412d-81ad-7082432dcb69'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.457442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] The token cache is being invalidate because project b1dafba5e88644db899a8d5ab9584c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:07.457820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.459246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-987cedba-33e5-412d-81ad-7082432dcb69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.461131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:07.461463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1069/2147] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:07 2022] DELETE /identity/v3/projects/b1dafba5e88644db899a8d5ab9584c1e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:07.469559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd1856-5727-4d9e-b89b-fbb8fac91b5f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.469927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd1856-5727-4d9e-b89b-fbb8fac91b5f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.471866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd1856-5727-4d9e-b89b-fbb8fac91b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.496904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-11fd1856-5727-4d9e-b89b-fbb8fac91b5f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:07.498260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-11fd1856-5727-4d9e-b89b-fbb8fac91b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.499005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.499744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.500100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.500370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.501027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.502765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.507563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1070/2148] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:07.515744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1079/2149] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:07.516509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579083cd-9c52-48ca-84a4-eb1083427c21 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.516873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579083cd-9c52-48ca-84a4-eb1083427c21 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.517208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579083cd-9c52-48ca-84a4-eb1083427c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.518923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.521511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.536135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffecbea-2846-4b47-bb85-a5a353a0722a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.541295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6830d8e-9688-4ba2-899f-3060bdd2d86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.542065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6830d8e-9688-4ba2-899f-3060bdd2d86b admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.542368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6830d8e-9688-4ba2-899f-3060bdd2d86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.542567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6830d8e-9688-4ba2-899f-3060bdd2d86b admin admin] PATH_INFO: `/v3/projects/e6434bbd301647eea243a1ef79bf1df2/users/433982506c6748c881794513ebd65858/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.547833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-579083cd-9c52-48ca-84a4-eb1083427c21 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:07.549049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-579083cd-9c52-48ca-84a4-eb1083427c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.556959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1071/2150] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:07.558387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6830d8e-9688-4ba2-899f-3060bdd2d86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6434bbd301647eea243a1ef79bf1df2, user_id=433982506c6748c881794513ebd65858, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.561054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579083cd-9c52-48ca-84a4-eb1083427c21 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.561510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6830d8e-9688-4ba2-899f-3060bdd2d86b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.570572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579083cd-9c52-48ca-84a4-eb1083427c21 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.576419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1080/2151] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:07 2022] PUT /identity/v3/projects/e6434bbd301647eea243a1ef79bf1df2/users/433982506c6748c881794513ebd65858/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:07.587710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246abe-3cf8-48ea-a8cf-d823eb81aa98 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.588068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246abe-3cf8-48ea-a8cf-d823eb81aa98 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.588345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246abe-3cf8-48ea-a8cf-d823eb81aa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.617276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-46246abe-3cf8-48ea-a8cf-d823eb81aa98 None None] MFA Rules not processed for user `433982506c6748c881794513ebd65858`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:07.618456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-46246abe-3cf8-48ea-a8cf-d823eb81aa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.627583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.628233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.628508 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.628806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.629442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:07.629745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.631331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.642594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1072/2152] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:40:07.646881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.656332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7294e4e-fea0-4202-9f00-cc1d7971a827 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.660925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.661517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.661788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.662054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.662677 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.664621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.691631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1081/2153] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:07.695848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1073/2154] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:40:07.696878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46246abe-3cf8-48ea-a8cf-d823eb81aa98 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.703615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.705298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:07.708578 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46246abe-3cf8-48ea-a8cf-d823eb81aa98 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.765227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097206c3-7cb7-4a28-a4e2-00b273447cc2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.767819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.768441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.768719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.768918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.769062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.769572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.769707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.769961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.770231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.771564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.798670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1082/2155] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:40:07.799688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.809956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9511c78-3c57-46c2-9cd2-09b5ca09d5a3 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.814094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:07.814475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.815090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.815360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.815662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.816566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.817930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.829111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1083/2156] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:07.833814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.835260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.836745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.844244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b19d7e-13c0-4c72-82d7-fc94c39c8418 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.846820 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1074/2157] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:40:07.848957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.849550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.849826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.850098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.850735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.852387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.868091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1084/2158] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:07.869414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.869551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:07.875142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:07.880907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4820f-ba54-43c7-b861-17249b8f5ec0 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.883403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315aa39-6df7-4577-a4b0-4a8b1ada6ba7 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:07.887036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff955fb-923e-4198-bfc9-59d6f190d452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.889102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:07.889750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.890033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.890300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] PATH_INFO: `/v3/users/433982506c6748c881794513ebd65858` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.891393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff955fb-923e-4198-bfc9-59d6f190d452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.891569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=433982506c6748c881794513ebd65858)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.891707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff955fb-923e-4198-bfc9-59d6f190d452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.891949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff955fb-923e-4198-bfc9-59d6f190d452 admin admin] PATH_INFO: `/v3/projects/110fa73293ea478e8fdc8819c9809cd8/users/e098fd4d56704a6cb0ba1d443fc41faa/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.893402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.919775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff955fb-923e-4198-bfc9-59d6f190d452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110fa73293ea478e8fdc8819c9809cd8, user_id=e098fd4d56704a6cb0ba1d443fc41faa, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:07.922421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff955fb-923e-4198-bfc9-59d6f190d452 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:07.943237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1085/2159] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:07 2022] PUT /identity/v3/projects/110fa73293ea478e8fdc8819c9809cd8/users/e098fd4d56704a6cb0ba1d443fc41faa/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:07.952914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433982506c6748c881794513ebd65858'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.953642 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433982506c6748c881794513ebd65858'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.953846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:07.954133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:07.954407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:07.954516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433982506c6748c881794513ebd65858'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.964468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433982506c6748c881794513ebd65858'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.973620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-8f175adb-0d58-4089-a338-52bbe3947dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433982506c6748c881794513ebd65858'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:07.998746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] MFA Rules not processed for user `e098fd4d56704a6cb0ba1d443fc41faa`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:08.000091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:08.003190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1075/2160] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:07 2022] DELETE /identity/v3/users/433982506c6748c881794513ebd65858 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:08.007895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.008189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.008456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.035287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:08.036534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:08.085259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1086/2161] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:08.096989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:08.098350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:08.102571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1076/2162] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:08.109276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:08.110660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:08.154927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf616b-06e1-4784-b1d7-7c5b43b87fd6 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:08.158702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:08.159314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.159593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.159883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.167589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaba2e-9c9e-44ca-9793-7a7594e3c053 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:08.168463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:08.169936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:08.171280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfef4c-e0b7-4eb0-8a24-8109a16f2e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:08.171936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfef4c-e0b7-4eb0-8a24-8109a16f2e6c service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.172203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfef4c-e0b7-4eb0-8a24-8109a16f2e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.172473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfef4c-e0b7-4eb0-8a24-8109a16f2e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.178456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:08.178797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1087/2163] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:40:08.184021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfef4c-e0b7-4eb0-8a24-8109a16f2e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:08.185495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfef4c-e0b7-4eb0-8a24-8109a16f2e6c service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:08.196904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1077/2164] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:40:08.497270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:08.501313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:08.531863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa179b-5135-4c57-9f60-0f7c89451c6c service glance] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:08.536705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:08.537346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.537644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.537911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] PATH_INFO: `/v3/projects/e6434bbd301647eea243a1ef79bf1df2` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.539084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6434bbd301647eea243a1ef79bf1df2)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:08.541349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:08.599006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6434bbd301647eea243a1ef79bf1df2', 'request_id': 'req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:08.606752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6434bbd301647eea243a1ef79bf1df2', 'request_id': 'req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:08.626803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6434bbd301647eea243a1ef79bf1df2', 'request_id': 'req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:08.626803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:08.645111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] The token cache is being invalidate because project e6434bbd301647eea243a1ef79bf1df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:08.645612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:08.646438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1f560ef9-ad42-4a0d-a3f8-255aa9fe1f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:08.648777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1088/2165] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:08 2022] DELETE /identity/v3/projects/e6434bbd301647eea243a1ef79bf1df2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:08.653749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9e77-15e1-4e8e-941f-95504c119e44 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.654039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9e77-15e1-4e8e-941f-95504c119e44 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.654315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9e77-15e1-4e8e-941f-95504c119e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.688107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-dafb9e77-15e1-4e8e-941f-95504c119e44 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:08.689469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-dafb9e77-15e1-4e8e-941f-95504c119e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:08.771340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1078/2166] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:08.779598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd895291-55ca-40d0-bac4-46092f56fc6b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.780066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd895291-55ca-40d0-bac4-46092f56fc6b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.780154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd895291-55ca-40d0-bac4-46092f56fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.799682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-dd895291-55ca-40d0-bac4-46092f56fc6b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:08.800892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-dd895291-55ca-40d0-bac4-46092f56fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:08.809205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1089/2167] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:08.814501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9e77-15e1-4e8e-941f-95504c119e44 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:08.830033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9e77-15e1-4e8e-941f-95504c119e44 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:08.836863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:08.837779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.838232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.838789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.839887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:08.840382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:08.842184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:08.857940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1079/2168] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:40:08.864627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd895291-55ca-40d0-bac4-46092f56fc6b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:08.877206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd895291-55ca-40d0-bac4-46092f56fc6b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:08.883298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:08.884247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:08.884247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:08.884457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:08.885101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:08.887985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:08.920311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1090/2169] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:08 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:08.924440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:08.935178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a28a9-2110-414e-9843-4e5b36eb5698 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:09.011135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:09.012384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:09.012658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:09.012932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:09.013569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:09.015652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:09.046470 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1080/2170] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:08 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:09.054560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:09.064768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42735-3280-47ad-923d-af54ec847b6f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:09.069882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:09.070783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:09.071118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:09.071383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:09.072074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:09.073750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:09.084377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1091/2171] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:09.088727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:09.099964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afff21-5764-41dc-9560-6beb5b9c893f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:09.105141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1ccf6-689a-400f-9306-bba7b32277e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:09.105762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1ccf6-689a-400f-9306-bba7b32277e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:09.106039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1ccf6-689a-400f-9306-bba7b32277e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:09.106307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1ccf6-689a-400f-9306-bba7b32277e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:09.106951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1ccf6-689a-400f-9306-bba7b32277e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:09.108631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1ccf6-689a-400f-9306-bba7b32277e8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:09.120955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1081/2172] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:09.125273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:09.135795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13975e2-bedc-45c2-8d02-3bfcd56bba53 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:09.140683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:09.141360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:09.141634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:09.141967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] PATH_INFO: `/v3/projects/fed55d34160e49fc9ae0deb6ac131c8f/users/a1377e1c75874545b4c6b472164e474a/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:09.161184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fed55d34160e49fc9ae0deb6ac131c8f, user_id=a1377e1c75874545b4c6b472164e474a, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:09.164482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:09.187063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1092/2173] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:09 2022] PUT /identity/v3/projects/fed55d34160e49fc9ae0deb6ac131c8f/users/a1377e1c75874545b4c6b472164e474a/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:09.198208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:09.198494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:09.198767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:09.228947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] MFA Rules not processed for user `a1377e1c75874545b4c6b472164e474a`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:09.230219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:09.278753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:09.299730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1082/2174] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:09.314433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:09.317348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:09.380522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2719c6-e873-4691-9a62-9bbf2e408a88 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:09.384520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaaef0-cf49-4bba-884b-a21b38a4b541 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:09.385130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaaef0-cf49-4bba-884b-a21b38a4b541 service glance] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:09.385917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaaef0-cf49-4bba-884b-a21b38a4b541 service glance] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:09.386182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaaef0-cf49-4bba-884b-a21b38a4b541 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:09.394344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaaef0-cf49-4bba-884b-a21b38a4b541 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:09.395846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaaef0-cf49-4bba-884b-a21b38a4b541 service glance] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:09.405399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1093/2175] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:16.358868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:16.360005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:16.456313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076674e-4f2c-434d-8177-ce28db5a5e30 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:16.462870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:16.463527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:16.463829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:16.464103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] PATH_INFO: `/v3/users/704306c49c4f499c80d9f3e9d2cd8d07` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:16.465280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=704306c49c4f499c80d9f3e9d2cd8d07)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:16.468272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:16.529694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '704306c49c4f499c80d9f3e9d2cd8d07'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.530356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '704306c49c4f499c80d9f3e9d2cd8d07'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.530956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '704306c49c4f499c80d9f3e9d2cd8d07'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.540831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '704306c49c4f499c80d9f3e9d2cd8d07'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.551624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '704306c49c4f499c80d9f3e9d2cd8d07'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.584560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1083/2176] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:16 2022] DELETE /identity/v3/users/704306c49c4f499c80d9f3e9d2cd8d07 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:16.588886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:16.589374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:16.589451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:16.612173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:16.613499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:16.698658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1094/2177] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:16.708587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:16.710050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:16.774727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec339f6-d337-4d6c-b208-eb5c4125caf3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:16.778483 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7411a-ccbf-4340-b9ab-d51842aee673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:16.779440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7411a-ccbf-4340-b9ab-d51842aee673 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:16.779757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7411a-ccbf-4340-b9ab-d51842aee673 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:16.780032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7411a-ccbf-4340-b9ab-d51842aee673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:16.790197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7411a-ccbf-4340-b9ab-d51842aee673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:16.792488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7411a-ccbf-4340-b9ab-d51842aee673 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:16.803828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1084/2178] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:16.820448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:16.823548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:16.838504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4832f84-4192-4d99-9dee-413dc06ef64f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:16.846016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:16.847042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:16.847439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:16.847843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] PATH_INFO: `/v3/users/9e68e99d38d0457292dcfbd240dbe041` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:16.849194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e68e99d38d0457292dcfbd240dbe041)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:16.851257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:16.920098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:16.949968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e68e99d38d0457292dcfbd240dbe041'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.951369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e68e99d38d0457292dcfbd240dbe041'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.971310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e68e99d38d0457292dcfbd240dbe041'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.972761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e68e99d38d0457292dcfbd240dbe041'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:16.985649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e68e99d38d0457292dcfbd240dbe041'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:17.018114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1095/2179] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:16 2022] DELETE /identity/v3/users/9e68e99d38d0457292dcfbd240dbe041 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:17.025011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:17.025464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:17.026040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:17.073708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:17.075019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:17.083200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:17.182732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1085/2180] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:17.220147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:17.222013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:17.226319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:17.226319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:17.243984 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8110e-9584-40ca-800b-c36b3761ee29 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:17.250226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:17.250506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:17.250843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:17.251211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] PATH_INFO: `/v3/projects/d588ef952640489f852a4e1580794f1f` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:17.252817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d588ef952640489f852a4e1580794f1f)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:17.254961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:17.319385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c01d91-f56c-4e8a-9b49-e361e86b25b3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:17.333421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd588ef952640489f852a4e1580794f1f', 'request_id': 'req-8b9e2be6-eae9-4788-abdd-8f9877085883'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:17.340069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd588ef952640489f852a4e1580794f1f', 'request_id': 'req-8b9e2be6-eae9-4788-abdd-8f9877085883'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:17.347433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd588ef952640489f852a4e1580794f1f', 'request_id': 'req-8b9e2be6-eae9-4788-abdd-8f9877085883'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:17.372702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] The token cache is being invalidate because project d588ef952640489f852a4e1580794f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:17.373559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:17.377061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b9e2be6-eae9-4788-abdd-8f9877085883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:17.384614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1096/2181] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:17 2022] DELETE /identity/v3/projects/d588ef952640489f852a4e1580794f1f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:17.386598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48f37-9bdd-4e02-9d31-978cf5c18156 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:17.388514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c8dc3-d376-4d43-a4b3-f3daabc90a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:17.389284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c8dc3-d376-4d43-a4b3-f3daabc90a1c service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:17.389469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c8dc3-d376-4d43-a4b3-f3daabc90a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:17.389566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48f37-9bdd-4e02-9d31-978cf5c18156 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:17.389957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c8dc3-d376-4d43-a4b3-f3daabc90a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:17.390246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48f37-9bdd-4e02-9d31-978cf5c18156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:17.392541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cc3c8dc3-d376-4d43-a4b3-f3daabc90a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:17.477702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-05f48f37-9bdd-4e02-9d31-978cf5c18156 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:17.478965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-05f48f37-9bdd-4e02-9d31-978cf5c18156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:17.548619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c8dc3-d376-4d43-a4b3-f3daabc90a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:17.550229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c8dc3-d376-4d43-a4b3-f3daabc90a1c service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:17.572782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1086/2182] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:17.634738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1097/2183] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:17.636474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66069c6a-675b-4afe-9804-f16d9ffd5d5f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:17.636753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66069c6a-675b-4afe-9804-f16d9ffd5d5f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:17.637031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66069c6a-675b-4afe-9804-f16d9ffd5d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:17.664120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-66069c6a-675b-4afe-9804-f16d9ffd5d5f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:17.665258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-66069c6a-675b-4afe-9804-f16d9ffd5d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:17.688272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1087/2184] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:17.693083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f48f37-9bdd-4e02-9d31-978cf5c18156 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:17.704086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f48f37-9bdd-4e02-9d31-978cf5c18156 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:17.720123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:17.721136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:17.721652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:17.722536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:17.723540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:17.724298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:17.726509 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:17.749613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1098/2185] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:17.750981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66069c6a-675b-4afe-9804-f16d9ffd5d5f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:17.765253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66069c6a-675b-4afe-9804-f16d9ffd5d5f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:17.770682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:17.771312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:17.771601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:17.771890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:17.774239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:17.779325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:17.803840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:17.867903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1088/2186] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:17 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:17.873619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:17.886870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c87f5-058d-4525-93aa-3018f27a3bef admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:17.920973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:18.055844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.056253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.056534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.056807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.057452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.059403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.093655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.095716 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.118342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1099/2187] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:17 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:18.119816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.122443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178480ae-ec81-4921-a7b8-8b9e7fc6f692 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.128919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.129590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.129918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.130193 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] PATH_INFO: `/v3/projects/f503ef98fdaf4279b0013e5c1f529e57` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.131337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=f503ef98fdaf4279b0013e5c1f529e57)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.133354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.146333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a236418-d7c7-4ba1-9a73-810e2889885d admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.165515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.166396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.166751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.167145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.167860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.169854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.183874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1100/2188] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:18.189067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.200185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8587c-1a60-4815-b0ac-997ec5e0ce25 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.215726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f503ef98fdaf4279b0013e5c1f529e57', 'request_id': 'req-3579bc10-a330-4654-9fad-468c80d3d0af'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.224260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f503ef98fdaf4279b0013e5c1f529e57', 'request_id': 'req-3579bc10-a330-4654-9fad-468c80d3d0af'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.234902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f503ef98fdaf4279b0013e5c1f529e57', 'request_id': 'req-3579bc10-a330-4654-9fad-468c80d3d0af'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.254248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] The token cache is being invalidate because project f503ef98fdaf4279b0013e5c1f529e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:18.254540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.255221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.256810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1089/2189] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:18 2022] DELETE /identity/v3/projects/f503ef98fdaf4279b0013e5c1f529e57 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:18.260815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.262184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.269332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.269957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.270244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.270533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.271187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.273003 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.287758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1101/2190] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:18.290782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.292785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.306086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a76021-22b9-44ba-8774-d4598b97c075 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.311901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe03cd-b094-4c9d-aecf-b0f77088841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.312642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe03cd-b094-4c9d-aecf-b0f77088841d admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.312913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe03cd-b094-4c9d-aecf-b0f77088841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.313182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe03cd-b094-4c9d-aecf-b0f77088841d admin admin] PATH_INFO: `/v3/projects/375a7cf8cbc042a187f2b144d907c175/users/bf99219df4fb45a4b7ec5fea537f8be2/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.331366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe03cd-b094-4c9d-aecf-b0f77088841d admin admin] RBAC: Authorizing `identity:create_grant(project_id=375a7cf8cbc042a187f2b144d907c175, user_id=bf99219df4fb45a4b7ec5fea537f8be2, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.334259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe03cd-b094-4c9d-aecf-b0f77088841d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.347540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579bc10-a330-4654-9fad-468c80d3d0af admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.353114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1102/2191] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:18 2022] PUT /identity/v3/projects/375a7cf8cbc042a187f2b144d907c175/users/bf99219df4fb45a4b7ec5fea537f8be2/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:18.380115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a6007-2c54-47a6-b6de-c7315e523d40 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.381178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a6007-2c54-47a6-b6de-c7315e523d40 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.381597 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a6007-2c54-47a6-b6de-c7315e523d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.411458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b52a6007-2c54-47a6-b6de-c7315e523d40 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:18.412719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b52a6007-2c54-47a6-b6de-c7315e523d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.446028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.446688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.446961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.447225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] PATH_INFO: `/v3/users/0bd05d8e060944398af586951e761641` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.448428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd05d8e060944398af586951e761641)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.450380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.498802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:18.518651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1103/2192] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:18.520123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7fee-ab5b-48c2-af11-4b749770255c None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.520496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7fee-ab5b-48c2-af11-4b749770255c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.520902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7fee-ab5b-48c2-af11-4b749770255c None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.522591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1104/2193] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:40:18.535744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd05d8e060944398af586951e761641'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.535956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7fee-ab5b-48c2-af11-4b749770255c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.536836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd05d8e060944398af586951e761641'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.537584 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd05d8e060944398af586951e761641'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.538047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-858d7fee-ab5b-48c2-af11-4b749770255c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.551376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd05d8e060944398af586951e761641'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.552875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:18.561480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-702a19ad-305f-4b11-ba5d-740a05171507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd05d8e060944398af586951e761641'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.598782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1090/2194] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:18 2022] DELETE /identity/v3/users/0bd05d8e060944398af586951e761641 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:18.600628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024138-3549-4b0b-9385-e2535910084a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.600628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024138-3549-4b0b-9385-e2535910084a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.601001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024138-3549-4b0b-9385-e2535910084a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.628476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5c024138-3549-4b0b-9385-e2535910084a None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:18.629957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5c024138-3549-4b0b-9385-e2535910084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.637960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7fee-ab5b-48c2-af11-4b749770255c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.718716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.719373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.719685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.719961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.721724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.737163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1091/2195] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:40:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:18.738091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c024138-3549-4b0b-9385-e2535910084a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.739403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5c024138-3549-4b0b-9385-e2535910084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.813268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.815249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.829170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c024138-3549-4b0b-9385-e2535910084a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.830285 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1105/2196] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:18.831186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.833022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.844786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.846729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.847046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.847421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] PATH_INFO: `/v3/users/904630cc2e60419d88afd9cfc1c5914d` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.849277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=904630cc2e60419d88afd9cfc1c5914d)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.851517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.858485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612e002-345c-4f55-a9e4-d6ec0fea4f1f service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.879291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.880207 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.880657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.881191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.891781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:18.894031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:18.923615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1106/2197] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:18.928612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:18.940607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '904630cc2e60419d88afd9cfc1c5914d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.941531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '904630cc2e60419d88afd9cfc1c5914d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.943253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '904630cc2e60419d88afd9cfc1c5914d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.955622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:18.957142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:18.980615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '904630cc2e60419d88afd9cfc1c5914d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.989017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fb410-928e-4237-821b-f21280907c58 service placement] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:18.991922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-940c99ff-884f-4b50-b29c-aa62e750eff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '904630cc2e60419d88afd9cfc1c5914d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:18.996080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:18.997200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:18.997200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:18.997775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] PATH_INFO: `/v3/projects/78cde154780549a8b096600953736bb3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:18.999242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=78cde154780549a8b096600953736bb3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.001194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.012428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1092/2198] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:18 2022] DELETE /identity/v3/users/904630cc2e60419d88afd9cfc1c5914d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:19.059497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78cde154780549a8b096600953736bb3', 'request_id': 'req-bb734586-1564-4453-b20b-eb55c6ed7c2d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.064602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:19.075516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78cde154780549a8b096600953736bb3', 'request_id': 'req-bb734586-1564-4453-b20b-eb55c6ed7c2d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.100420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78cde154780549a8b096600953736bb3', 'request_id': 'req-bb734586-1564-4453-b20b-eb55c6ed7c2d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.126584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] The token cache is being invalidate because project 78cde154780549a8b096600953736bb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:19.126805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.129801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bb734586-1564-4453-b20b-eb55c6ed7c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.135628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1107/2199] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:18 2022] DELETE /identity/v3/projects/78cde154780549a8b096600953736bb3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:19.139250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc24973-7f87-4c0f-b4cf-a005cd096de3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.139424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc24973-7f87-4c0f-b4cf-a005cd096de3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.139706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc24973-7f87-4c0f-b4cf-a005cd096de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.161783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ccc24973-7f87-4c0f-b4cf-a005cd096de3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:19.163688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ccc24973-7f87-4c0f-b4cf-a005cd096de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:19.258184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1093/2200] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:19.259767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.260077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.260344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.305182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:19.306523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:19.339276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1108/2201] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:19.340688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc24973-7f87-4c0f-b4cf-a005cd096de3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.352121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc24973-7f87-4c0f-b4cf-a005cd096de3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.357014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.357236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.357672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.357946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.358214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.359075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:19.359380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.360086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:19.361016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.377925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1094/2202] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:19.393020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.398998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321c80b-bd8f-41f7-b959-1e2ad23094f9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.408142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.409473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.410221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.411241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] PATH_INFO: `/v3/projects/822f0c2e66354549aeb13d73c512b09d` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.412592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=822f0c2e66354549aeb13d73c512b09d)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.413463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017468b0-a251-49b1-bb94-638af2a51be3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.415422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.420413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.420628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.421307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.421604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.422250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.424235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.473824 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1095/2203] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:19.479787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.501806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822f0c2e66354549aeb13d73c512b09d', 'request_id': 'req-dfa6cb36-02bf-4b72-b684-e447a2778b4f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.515747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822f0c2e66354549aeb13d73c512b09d', 'request_id': 'req-dfa6cb36-02bf-4b72-b684-e447a2778b4f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.523822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822f0c2e66354549aeb13d73c512b09d', 'request_id': 'req-dfa6cb36-02bf-4b72-b684-e447a2778b4f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.561023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48ff4-cb90-462f-b406-6b5fe0b1b2cb admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.569313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] The token cache is being invalidate because project 822f0c2e66354549aeb13d73c512b09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:19.569811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.570718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dfa6cb36-02bf-4b72-b684-e447a2778b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:19.574056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1109/2204] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:19 2022] DELETE /identity/v3/projects/822f0c2e66354549aeb13d73c512b09d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:40:19.582479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f203e90-5c78-44e7-b24b-36f6699b4c59 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.583281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f203e90-5c78-44e7-b24b-36f6699b4c59 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.583982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f203e90-5c78-44e7-b24b-36f6699b4c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.587910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:19.615190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9f203e90-5c78-44e7-b24b-36f6699b4c59 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:19.616790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9f203e90-5c78-44e7-b24b-36f6699b4c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:19.646933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.648339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.649404 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.650366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.651950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.654661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.693746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1096/2205] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:40:19.699659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.703435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:19.715759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1110/2206] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:19.725582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94f0a5-31f2-4730-8678-6e1c7f87b5fe None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.726146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94f0a5-31f2-4730-8678-6e1c7f87b5fe None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.726490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94f0a5-31f2-4730-8678-6e1c7f87b5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.732244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b08b1-cde7-4e5d-b206-18cbc84f8b95 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.744831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.746098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.746588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.746961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.747889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.751239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.764467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-aa94f0a5-31f2-4730-8678-6e1c7f87b5fe None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:19.765752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-aa94f0a5-31f2-4730-8678-6e1c7f87b5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:19.769174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1097/2207] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:19.770199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.780740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444896b8-dee8-46f2-a32f-65da47a85c6c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.785899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.786368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1111/2208] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:19.786519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.786827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.787111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.787762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.789107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228fa5b-139c-4f6e-89e7-340fb20a24d0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.789638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228fa5b-139c-4f6e-89e7-340fb20a24d0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.790014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.790343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228fa5b-139c-4f6e-89e7-340fb20a24d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.801591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1098/2209] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:19.805467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.817037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8848e-0df2-478f-9aad-fc5e55cc7801 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.822912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.823479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.823933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.824267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.824365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9228fa5b-139c-4f6e-89e7-340fb20a24d0 None None] MFA Rules not processed for user `bf99219df4fb45a4b7ec5fea537f8be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:19.825016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:19.825412 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.825557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9228fa5b-139c-4f6e-89e7-340fb20a24d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:19.827421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.849333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1099/2210] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:19.850290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.861305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3ea7-ca5f-4a6b-9b47-7de21430a9dc admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:19.867552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:19.872075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.872577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.873004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] PATH_INFO: `/v3/projects/e94077f04cc548c49f5a39ce0fabff3e/users/af5c2bdfab9144529aaed3d8b6cd376d/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.893343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:19.896836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94077f04cc548c49f5a39ce0fabff3e, user_id=af5c2bdfab9144529aaed3d8b6cd376d, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:19.901123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:19.916707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1100/2211] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:19 2022] PUT /identity/v3/projects/e94077f04cc548c49f5a39ce0fabff3e/users/af5c2bdfab9144529aaed3d8b6cd376d/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:19.917603 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:19.931953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1112/2212] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:19.933428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b09ccc-40b4-4194-8447-9bec6542616b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:19.933818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b09ccc-40b4-4194-8447-9bec6542616b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:19.934194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b09ccc-40b4-4194-8447-9bec6542616b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:19.974861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-40b09ccc-40b4-4194-8447-9bec6542616b None None] MFA Rules not processed for user `af5c2bdfab9144529aaed3d8b6cd376d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:19.976127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-40b09ccc-40b4-4194-8447-9bec6542616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:20.006190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1203-490f-4014-9bf0-b5a0905ebd41 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.011336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.011995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.012274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.012542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.013223 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.015139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.043866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1101/2213] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:20.045232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.045961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1113/2214] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:20.046985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:20.047977 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b09ccc-40b4-4194-8447-9bec6542616b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.064622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b09ccc-40b4-4194-8447-9bec6542616b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.102883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:20.115375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ece47-4960-40f9-8758-b9ef3c1e88eb admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.119261 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.119900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.120174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.120437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.124230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.125118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.125397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.125676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.126341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.128773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.154594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1114/2215] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:20 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:20.154594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.154594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:20.167478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b678d19-a5f5-4bec-8de3-7a6f99665da6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.172282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.172907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.173187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.173464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.190158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.191668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.200806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1102/2216] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:20.201858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.212156 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315b210-6e6e-4252-a4dd-db002b30900a service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.217120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.218084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.218360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.218626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.219264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.220878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.235179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1103/2217] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:20.236088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.247290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f526f-ae49-4db2-9f40-1382735e821d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.248594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.251202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.252476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.253113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.253390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.253657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.254294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.256142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:20.256922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.262661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1115/2218] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:20.270506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1104/2219] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:20.287277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.288650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.298663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54026f66-74a3-4092-808a-9a168f9be3c2 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.299752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b7855-c723-4cf7-bd2e-b7c3b8f2b0e3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.304341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8a4f2-3b90-4eed-ac07-2fa09217b28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.305378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a4f2-3b90-4eed-ac07-2fa09217b28b admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.305764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a4f2-3b90-4eed-ac07-2fa09217b28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.306137 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a4f2-3b90-4eed-ac07-2fa09217b28b admin admin] PATH_INFO: `/v3/projects/e27e3216211942959311372c2d078202/users/2864050fd1a64646a8760f0172be8f4a/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.312040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.312692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.312973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.313252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] PATH_INFO: `/v3/users/af5c2bdfab9144529aaed3d8b6cd376d` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.314411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5c2bdfab9144529aaed3d8b6cd376d)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.316833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.325070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8a4f2-3b90-4eed-ac07-2fa09217b28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27e3216211942959311372c2d078202, user_id=2864050fd1a64646a8760f0172be8f4a, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.328412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8a4f2-3b90-4eed-ac07-2fa09217b28b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.347849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1116/2220] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:20 2022] PUT /identity/v3/projects/e27e3216211942959311372c2d078202/users/2864050fd1a64646a8760f0172be8f4a/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:20.353707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.354527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.354829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.380545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:20.381785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:20.406808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5c2bdfab9144529aaed3d8b6cd376d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:20.407947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5c2bdfab9144529aaed3d8b6cd376d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:20.409151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5c2bdfab9144529aaed3d8b6cd376d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:20.418807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5c2bdfab9144529aaed3d8b6cd376d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:20.428354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1a37e19f-5aab-4a5a-9311-f2703ee0a160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5c2bdfab9144529aaed3d8b6cd376d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:20.448925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1105/2221] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:20 2022] DELETE /identity/v3/users/af5c2bdfab9144529aaed3d8b6cd376d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:20.455242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.456017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.456840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.471109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1117/2222] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:20.482398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.483899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:20.493344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:20.494715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:20.580464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:20.601973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202777c3-7ca1-412e-934e-69a9fd7dc900 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.654839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1106/2223] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:20.665897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:20.667332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:20.693851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.694407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.694617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.694829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.712464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.714113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.724974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1118/2224] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:40:20.728960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b71288-bd10-47ce-82ee-78faef6e5748 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:20.732659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231342c-fdcc-46c3-9884-0267c44fdceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:20.733329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231342c-fdcc-46c3-9884-0267c44fdceb service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:20.733604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231342c-fdcc-46c3-9884-0267c44fdceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:20.733871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231342c-fdcc-46c3-9884-0267c44fdceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:20.749199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231342c-fdcc-46c3-9884-0267c44fdceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:20.751267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231342c-fdcc-46c3-9884-0267c44fdceb service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:20.762396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1107/2225] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:40:21.073157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:21.082382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:21.099425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41948367-02ed-4598-bf3e-837f854aa069 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:21.106994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752d322-90c9-4925-b392-748639547796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:21.107688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d322-90c9-4925-b392-748639547796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.107995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d322-90c9-4925-b392-748639547796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.108269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d322-90c9-4925-b392-748639547796 admin admin] PATH_INFO: `/v3/projects/e94077f04cc548c49f5a39ce0fabff3e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.109429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752d322-90c9-4925-b392-748639547796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94077f04cc548c49f5a39ce0fabff3e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:21.111370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752d322-90c9-4925-b392-748639547796 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:21.153297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:21.169251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0752d322-90c9-4925-b392-748639547796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94077f04cc548c49f5a39ce0fabff3e', 'request_id': 'req-0752d322-90c9-4925-b392-748639547796'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:21.178047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0752d322-90c9-4925-b392-748639547796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94077f04cc548c49f5a39ce0fabff3e', 'request_id': 'req-0752d322-90c9-4925-b392-748639547796'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:21.186672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0752d322-90c9-4925-b392-748639547796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94077f04cc548c49f5a39ce0fabff3e', 'request_id': 'req-0752d322-90c9-4925-b392-748639547796'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:21.201727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0752d322-90c9-4925-b392-748639547796 admin admin] The token cache is being invalidate because project e94077f04cc548c49f5a39ce0fabff3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:21.202014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0752d322-90c9-4925-b392-748639547796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:21.202728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0752d322-90c9-4925-b392-748639547796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:21.204291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1119/2226] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:21 2022] DELETE /identity/v3/projects/e94077f04cc548c49f5a39ce0fabff3e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:21.209381 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6081247-9748-4a90-9c11-ad1ea14b1150 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.209663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6081247-9748-4a90-9c11-ad1ea14b1150 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.209932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6081247-9748-4a90-9c11-ad1ea14b1150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.229964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f6081247-9748-4a90-9c11-ad1ea14b1150 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:21.231679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f6081247-9748-4a90-9c11-ad1ea14b1150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:21.295160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1108/2227] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:21.303457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6837071-773d-49eb-9a11-316cb4b104e1 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.303893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6837071-773d-49eb-9a11-316cb4b104e1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.304233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6837071-773d-49eb-9a11-316cb4b104e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.329023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d6837071-773d-49eb-9a11-316cb4b104e1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:21.330261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d6837071-773d-49eb-9a11-316cb4b104e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:21.338511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1120/2228] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:21.343525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6081247-9748-4a90-9c11-ad1ea14b1150 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:21.353305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6081247-9748-4a90-9c11-ad1ea14b1150 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:21.358221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:21.358850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.359122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.359385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.360106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:21.360403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:21.362400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:21.376315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1109/2229] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:21.381216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6837071-773d-49eb-9a11-316cb4b104e1 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:21.394434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6837071-773d-49eb-9a11-316cb4b104e1 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:21.400053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:21.400830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.401229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.401716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.402364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:21.404817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:21.438817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1121/2230] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:21 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:21.443120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:21.454230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061677b9-5bab-4605-9888-abc2013e6be7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:21.537060 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:21.537740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.537953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.538264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.539169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:21.542037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:21.571792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1110/2231] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:21 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:21.576833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:21.589178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35be9f2-10cf-45d6-901b-f18f575a6a37 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:21.598042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:21.598815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.599226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.599761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.600525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:21.604368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:21.640850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1122/2232] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:21.642200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:21.667362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcabb7-8fe6-4c30-ab7d-431987b6e37c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:21.680400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c6da4-c6a2-44d7-8d69-a6b7b29c14a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:21.681194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c6da4-c6a2-44d7-8d69-a6b7b29c14a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.681480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c6da4-c6a2-44d7-8d69-a6b7b29c14a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.682479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c6da4-c6a2-44d7-8d69-a6b7b29c14a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.683766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c6da4-c6a2-44d7-8d69-a6b7b29c14a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:21.690147 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c6da4-c6a2-44d7-8d69-a6b7b29c14a9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:21.693957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:21.722638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1111/2233] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:21.733962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:21.740651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.740948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.741219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.757760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc06f2-1eb2-4641-9bdf-69955ef232ae admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:21.767886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b2c13-30cb-430c-bdbd-c9223b6a683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:21.768962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b2c13-30cb-430c-bdbd-c9223b6a683a admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.769469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b2c13-30cb-430c-bdbd-c9223b6a683a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.769939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b2c13-30cb-430c-bdbd-c9223b6a683a admin admin] PATH_INFO: `/v3/projects/1102a5e530fc48e8867e4c62f8415d34/users/42bfa0f5665b49fc91c24089c2eece1e/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.770796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] MFA Rules not processed for user `bf99219df4fb45a4b7ec5fea537f8be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:21.772260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:21.795682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b2c13-30cb-430c-bdbd-c9223b6a683a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1102a5e530fc48e8867e4c62f8415d34, user_id=42bfa0f5665b49fc91c24089c2eece1e, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:21.799825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b2c13-30cb-430c-bdbd-c9223b6a683a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:21.838058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1123/2234] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:21 2022] PUT /identity/v3/projects/1102a5e530fc48e8867e4c62f8415d34/users/42bfa0f5665b49fc91c24089c2eece1e/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:21.851681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372caa-a575-4c25-8e5d-e27b510c5ad5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:21.851681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372caa-a575-4c25-8e5d-e27b510c5ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:21.851681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372caa-a575-4c25-8e5d-e27b510c5ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:21.906863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-62372caa-a575-4c25-8e5d-e27b510c5ad5 None None] MFA Rules not processed for user `42bfa0f5665b49fc91c24089c2eece1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:21.908323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-62372caa-a575-4c25-8e5d-e27b510c5ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:21.936076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1112/2235] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:22.027744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1124/2236] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:22.037731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:22.039280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:22.111926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342cae-5c6d-493b-9077-45c74cc3a9fb None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:22.181573 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03a8fe-c510-4f62-88af-1a2840b8a7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:22.182319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03a8fe-c510-4f62-88af-1a2840b8a7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:22.182922 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03a8fe-c510-4f62-88af-1a2840b8a7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:22.183349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03a8fe-c510-4f62-88af-1a2840b8a7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:22.195609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03a8fe-c510-4f62-88af-1a2840b8a7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:22.197247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03a8fe-c510-4f62-88af-1a2840b8a7e9 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:22.209176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1113/2237] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:25.075353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62372caa-a575-4c25-8e5d-e27b510c5ad5 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:25.144913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:25.171236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62372caa-a575-4c25-8e5d-e27b510c5ad5 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:25.179381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:25.180399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:25.180902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:25.181379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] PATH_INFO: `/v3/users/42bfa0f5665b49fc91c24089c2eece1e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:25.184308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] RBAC: Authorizing `identity:delete_user(user_id=42bfa0f5665b49fc91c24089c2eece1e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:25.187443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:25.278107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42bfa0f5665b49fc91c24089c2eece1e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:25.279266 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42bfa0f5665b49fc91c24089c2eece1e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:25.291525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42bfa0f5665b49fc91c24089c2eece1e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:25.292532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42bfa0f5665b49fc91c24089c2eece1e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:25.302235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42bfa0f5665b49fc91c24089c2eece1e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:25.334966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1125/2238] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:25 2022] DELETE /identity/v3/users/42bfa0f5665b49fc91c24089c2eece1e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:25.336721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a090d-ca30-40ab-8e3b-2606ed83cff4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:25.337518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a090d-ca30-40ab-8e3b-2606ed83cff4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:25.337955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a090d-ca30-40ab-8e3b-2606ed83cff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:25.364602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-df6a090d-ca30-40ab-8e3b-2606ed83cff4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:25.366766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-df6a090d-ca30-40ab-8e3b-2606ed83cff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:25.469036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:25.491176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1114/2239] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:25.534443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:25.536467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:25.630670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0da86-d12b-4e65-bf60-00e0b6bbefae admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:25.636476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:25.637265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:25.638007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:25.638418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:25.659614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:25.659855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:25.660572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:25.674915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:25.678305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:25.695039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] MFA Rules not processed for user `2864050fd1a64646a8760f0172be8f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:25.696624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:25.704851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1126/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:25.835112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1115/2241] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:25.859434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:25.861111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:26.034361 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:26.039812 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bcc1b-c458-4a79-83b1-6a99baf30a51 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.047941 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccefb73-9775-4c4a-b4ed-5a8f44a19145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.048056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefb73-9775-4c4a-b4ed-5a8f44a19145 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.048382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefb73-9775-4c4a-b4ed-5a8f44a19145 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.048595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefb73-9775-4c4a-b4ed-5a8f44a19145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.059095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccefb73-9775-4c4a-b4ed-5a8f44a19145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.061036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccefb73-9775-4c4a-b4ed-5a8f44a19145 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.073668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1127/2242] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:26.150286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:26.152607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:26.183698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bdb31-fec9-4248-aa00-9e5af19e4fd9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.202205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.202902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.203161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.203429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] PATH_INFO: `/v3/projects/1102a5e530fc48e8867e4c62f8415d34` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.205058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1102a5e530fc48e8867e4c62f8415d34)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.207014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.272231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1102a5e530fc48e8867e4c62f8415d34', 'request_id': 'req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:26.279343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1102a5e530fc48e8867e4c62f8415d34', 'request_id': 'req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:26.287083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1102a5e530fc48e8867e4c62f8415d34', 'request_id': 'req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:26.306028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] The token cache is being invalidate because project 1102a5e530fc48e8867e4c62f8415d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:26.306586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:26.307480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-91b1bfb4-ae7e-40a6-9fd8-9a19d26bf9d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:26.313822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1116/2243] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:26 2022] DELETE /identity/v3/projects/1102a5e530fc48e8867e4c62f8415d34 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:26.314342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f8322-2ccb-4059-a715-4bac994a353f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.314783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f8322-2ccb-4059-a715-4bac994a353f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.314918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f8322-2ccb-4059-a715-4bac994a353f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.338396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2e8f8322-2ccb-4059-a715-4bac994a353f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:26.339889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2e8f8322-2ccb-4059-a715-4bac994a353f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:26.436911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58645389-5751-4334-b805-bfb6fd442074 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.437455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58645389-5751-4334-b805-bfb6fd442074 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.437455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58645389-5751-4334-b805-bfb6fd442074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.440493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1128/2244] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:26.463678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-58645389-5751-4334-b805-bfb6fd442074 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:26.464903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-58645389-5751-4334-b805-bfb6fd442074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:26.473811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1117/2245] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:26.478192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f8322-2ccb-4059-a715-4bac994a353f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:26.490947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f8322-2ccb-4059-a715-4bac994a353f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.496672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.497643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.497927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.498210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.498891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:26.499185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.517874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.541306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1129/2246] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:26.548875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58645389-5751-4334-b805-bfb6fd442074 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:26.561145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58645389-5751-4334-b805-bfb6fd442074 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.575140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.575565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.575948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.576668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.577366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.579541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.618514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1118/2247] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:26 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:26.619779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:26.635600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13fab5-84f8-40e2-af0f-57b0e24f2a2e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.704831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.705462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.705744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.706020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.706656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.708781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.733877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1130/2248] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:26 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:26.738755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:26.757774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ee205-0991-4e21-a583-9c43ab93c61b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.763008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.763772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.763916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.764189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.764854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.766515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.779703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1119/2249] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:26.781962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:26.792725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045525d-e417-437d-a883-99a92ae59ec0 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.798293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80d838-9ee2-4867-a236-62b88a151d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.799023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80d838-9ee2-4867-a236-62b88a151d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.799368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80d838-9ee2-4867-a236-62b88a151d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.800147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80d838-9ee2-4867-a236-62b88a151d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.800453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80d838-9ee2-4867-a236-62b88a151d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.802803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80d838-9ee2-4867-a236-62b88a151d78 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.814889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1131/2250] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:26.819808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:26.831330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44de85-a080-4949-8ce1-a13238092001 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:26.847949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:26.848660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.848940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.849213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] PATH_INFO: `/v3/projects/80158ca668bd4ac28b6d438b752d82cc/users/baa7dabf86204ec2896fc192bcb8be5a/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.887355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80158ca668bd4ac28b6d438b752d82cc, user_id=baa7dabf86204ec2896fc192bcb8be5a, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:26.891940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:26.921854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1120/2251] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:26 2022] PUT /identity/v3/projects/80158ca668bd4ac28b6d438b752d82cc/users/baa7dabf86204ec2896fc192bcb8be5a/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:26.932721 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce16d-0d09-4fff-8cc7-70695addb732 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:26.933065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce16d-0d09-4fff-8cc7-70695addb732 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:26.933315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce16d-0d09-4fff-8cc7-70695addb732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:26.965476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-072ce16d-0d09-4fff-8cc7-70695addb732 None None] MFA Rules not processed for user `baa7dabf86204ec2896fc192bcb8be5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:26.966854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-072ce16d-0d09-4fff-8cc7-70695addb732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:27.024008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:27.050329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1132/2252] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:27.065153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:27.067182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:27.088111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:27.180152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d4acc-036a-4f05-98fd-ccf93b1562e4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:27.184219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa721de-78a5-4ee6-b341-5e32a54e07b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:27.184965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa721de-78a5-4ee6-b341-5e32a54e07b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:27.185303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa721de-78a5-4ee6-b341-5e32a54e07b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:27.187709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa721de-78a5-4ee6-b341-5e32a54e07b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:27.197971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa721de-78a5-4ee6-b341-5e32a54e07b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:27.200917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa721de-78a5-4ee6-b341-5e32a54e07b4 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:27.211423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1121/2253] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:27.244677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce16d-0d09-4fff-8cc7-70695addb732 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:27.350263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce16d-0d09-4fff-8cc7-70695addb732 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:27.356739 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:27.357406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:27.357685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:27.358022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] PATH_INFO: `/v3/users/baa7dabf86204ec2896fc192bcb8be5a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:27.363568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa7dabf86204ec2896fc192bcb8be5a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:27.365962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:27.446186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa7dabf86204ec2896fc192bcb8be5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:27.451151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa7dabf86204ec2896fc192bcb8be5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:27.460281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa7dabf86204ec2896fc192bcb8be5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:27.461085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa7dabf86204ec2896fc192bcb8be5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:27.470854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa7dabf86204ec2896fc192bcb8be5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:27.492823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1133/2254] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:27 2022] DELETE /identity/v3/users/baa7dabf86204ec2896fc192bcb8be5a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:27.497318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efb1e4-60c4-48e5-8bc2-ae1bc09a6a0d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:27.497843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efb1e4-60c4-48e5-8bc2-ae1bc09a6a0d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:27.497843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efb1e4-60c4-48e5-8bc2-ae1bc09a6a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:27.526009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-46efb1e4-60c4-48e5-8bc2-ae1bc09a6a0d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:27.527299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-46efb1e4-60c4-48e5-8bc2-ae1bc09a6a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:27.627952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1122/2255] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:27.634799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:27.638489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:27.719730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387cbf2-94f5-4a07-b09b-f21b7233a524 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:27.724386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:27.725183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:27.725599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:27.726001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:27.742902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:27.744890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:27.765450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1134/2256] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:27.858280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf0f1a-59da-4874-a40f-e3ab9a25e3cf None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:27.858398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf0f1a-59da-4874-a40f-e3ab9a25e3cf None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:27.858640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf0f1a-59da-4874-a40f-e3ab9a25e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:27.879254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c3cf0f1a-59da-4874-a40f-e3ab9a25e3cf None None] MFA Rules not processed for user `2864050fd1a64646a8760f0172be8f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:27.880501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c3cf0f1a-59da-4874-a40f-e3ab9a25e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:27.974956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1123/2257] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:28.650780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:28.652835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:28.666613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae43b90-4716-4abd-b71c-857fc65cc134 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:28.683135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:28.684875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:28.685419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:28.686354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] PATH_INFO: `/v3/projects/80158ca668bd4ac28b6d438b752d82cc` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:28.687967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80158ca668bd4ac28b6d438b752d82cc)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:28.690109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:28.693460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:28.765544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80158ca668bd4ac28b6d438b752d82cc', 'request_id': 'req-2861b3d4-99eb-49ad-b716-e63f5344d298'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:28.772683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80158ca668bd4ac28b6d438b752d82cc', 'request_id': 'req-2861b3d4-99eb-49ad-b716-e63f5344d298'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:28.780470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80158ca668bd4ac28b6d438b752d82cc', 'request_id': 'req-2861b3d4-99eb-49ad-b716-e63f5344d298'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:28.798666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] The token cache is being invalidate because project 80158ca668bd4ac28b6d438b752d82cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:28.799042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:28.799709 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2861b3d4-99eb-49ad-b716-e63f5344d298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:28.801462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1135/2258] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:28 2022] DELETE /identity/v3/projects/80158ca668bd4ac28b6d438b752d82cc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:28.806406 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dc0a5-8474-405d-a066-8d809e21fb25 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:28.806916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dc0a5-8474-405d-a066-8d809e21fb25 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:28.807013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dc0a5-8474-405d-a066-8d809e21fb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:28.827794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ed5dc0a5-8474-405d-a066-8d809e21fb25 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:28.829003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ed5dc0a5-8474-405d-a066-8d809e21fb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:28.893627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:28.914524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1124/2259] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:28.916519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed049a-676c-4caa-befc-2827f4014a9b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:28.917680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed049a-676c-4caa-befc-2827f4014a9b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:28.918053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed049a-676c-4caa-befc-2827f4014a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:28.952749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-aeed049a-676c-4caa-befc-2827f4014a9b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:28.954816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-aeed049a-676c-4caa-befc-2827f4014a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:28.975511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1136/2260] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:28.977279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dc0a5-8474-405d-a066-8d809e21fb25 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:28.988812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dc0a5-8474-405d-a066-8d809e21fb25 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:28.993394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:28.994129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:28.994407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:28.994757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:28.995418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:28.995833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:28.997565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:29.010981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1125/2261] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:29.016741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed049a-676c-4caa-befc-2827f4014a9b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:29.031494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed049a-676c-4caa-befc-2827f4014a9b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:29.042703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:29.043646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:29.044134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:29.044581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:29.045443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:29.047566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:29.117829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1137/2262] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:29 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:29.122276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:29.133033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31fd01-ce20-4a53-a3c5-31d6c20cdb08 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:29.261804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:29.262456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:29.262800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:29.276475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:29.280240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:29.282284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:29.328969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1126/2263] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:29 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:29.332098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:29.353768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1c77-67b3-495b-98d9-e641c6f4325c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:29.360896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:29.363710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:29.364004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:29.364281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:29.365025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:29.366970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:29.396139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1138/2264] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:40:29.404070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:29.422861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ded5c-9072-479b-a0ed-e858a8cea6b0 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:29.433242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efce221-2833-43d6-8c97-09a5fb7fd5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:29.434126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efce221-2833-43d6-8c97-09a5fb7fd5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:29.434290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efce221-2833-43d6-8c97-09a5fb7fd5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:29.434724 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efce221-2833-43d6-8c97-09a5fb7fd5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:29.435459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efce221-2833-43d6-8c97-09a5fb7fd5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:29.442303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efce221-2833-43d6-8c97-09a5fb7fd5c4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:29.468474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1127/2265] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:29.482732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:29.508345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08129467-07b1-4df2-8cd8-5c81ae4eec15 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:29.523471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:29.524401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:29.524829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:29.525247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] PATH_INFO: `/v3/projects/537c3a742b1c4bfaaa82bca4763c57cd/users/5e510fda5ca54080aa611a9c45506cad/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:29.566078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537c3a742b1c4bfaaa82bca4763c57cd, user_id=5e510fda5ca54080aa611a9c45506cad, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:29.570879 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:29.597849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1139/2266] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:29 2022] PUT /identity/v3/projects/537c3a742b1c4bfaaa82bca4763c57cd/users/5e510fda5ca54080aa611a9c45506cad/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:29.608874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179233ed-0c35-4379-b1be-0dd1fbb4a9b1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:29.616649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179233ed-0c35-4379-b1be-0dd1fbb4a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:29.616649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179233ed-0c35-4379-b1be-0dd1fbb4a9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:29.654091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-179233ed-0c35-4379-b1be-0dd1fbb4a9b1 None None] MFA Rules not processed for user `5e510fda5ca54080aa611a9c45506cad`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:29.655464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-179233ed-0c35-4379-b1be-0dd1fbb4a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:29.760318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1128/2267] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:29.776834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:29.779287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:29.877829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1b01-1bf0-4533-b9a4-4029bc6c64a6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:29.883825 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cd0a-e540-4d1d-82a9-ff560da99519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:29.885201 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd0a-e540-4d1d-82a9-ff560da99519 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:29.885532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd0a-e540-4d1d-82a9-ff560da99519 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:29.885916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd0a-e540-4d1d-82a9-ff560da99519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:29.905928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2cd0a-e540-4d1d-82a9-ff560da99519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:29.907663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2cd0a-e540-4d1d-82a9-ff560da99519 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:29.941068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1140/2268] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:39.756103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:39.928605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:44.790821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179233ed-0c35-4379-b1be-0dd1fbb4a9b1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:44.865432 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179233ed-0c35-4379-b1be-0dd1fbb4a9b1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:44.871057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:44.871708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:44.871985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:44.872255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] PATH_INFO: `/v3/users/5e510fda5ca54080aa611a9c45506cad` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:44.873539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e510fda5ca54080aa611a9c45506cad)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:44.875453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:44.927416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e510fda5ca54080aa611a9c45506cad'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:44.928257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e510fda5ca54080aa611a9c45506cad'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:44.928853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e510fda5ca54080aa611a9c45506cad'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:44.936500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e510fda5ca54080aa611a9c45506cad'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:44.943533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e510fda5ca54080aa611a9c45506cad'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:44.960280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1129/2269] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:44 2022] DELETE /identity/v3/users/5e510fda5ca54080aa611a9c45506cad => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:44.965002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:44.965466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:44.965558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:44.985949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:44.987171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:45.053347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1141/2270] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:45.063383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:45.065102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:45.128345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23df4e-2824-4cd6-b632-a047922ce1f6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:45.132104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:45.132717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:45.132993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:45.133265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:45.143610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:45.145068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:45.155449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1130/2271] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:45.446771 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:45.450252 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:45.464214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c084e9c-595b-4ce9-9dc6-a77be40f0f6f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:45.470985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:45.471911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:45.472548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:45.472901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] PATH_INFO: `/v3/projects/537c3a742b1c4bfaaa82bca4763c57cd` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:45.474883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] RBAC: Authorizing `identity:delete_project(project_id=537c3a742b1c4bfaaa82bca4763c57cd)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:45.477014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:45.541647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537c3a742b1c4bfaaa82bca4763c57cd', 'request_id': 'req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:45.548329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537c3a742b1c4bfaaa82bca4763c57cd', 'request_id': 'req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:45.554764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537c3a742b1c4bfaaa82bca4763c57cd', 'request_id': 'req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:45.569963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] The token cache is being invalidate because project 537c3a742b1c4bfaaa82bca4763c57cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:45.570513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:45.571601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-4117d7e5-f191-45ae-ad0a-0a0e65b8262c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:45.574032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1142/2272] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:45 2022] DELETE /identity/v3/projects/537c3a742b1c4bfaaa82bca4763c57cd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:54.221260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:54.222845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:54.236378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fadd85-4991-4c22-aa39-c7cd7ac22257 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:54.255197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:54.304430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:54.305070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:54.305344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:54.305612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] PATH_INFO: `/v3/users/a1377e1c75874545b4c6b472164e474a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:54.306758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1377e1c75874545b4c6b472164e474a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:54.308849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:54.359521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1377e1c75874545b4c6b472164e474a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:54.360219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1377e1c75874545b4c6b472164e474a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:54.360836 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1377e1c75874545b4c6b472164e474a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:54.369917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1377e1c75874545b4c6b472164e474a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:54.377263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1377e1c75874545b4c6b472164e474a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:54.397348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1131/2273] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:40:54 2022] DELETE /identity/v3/users/a1377e1c75874545b4c6b472164e474a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:54.402009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:54.402497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:54.402626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:54.423287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:54.424543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:54.496850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1143/2274] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:54.507348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:54.509340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:54.626686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb97b7-69f8-4079-9f01-99aa38e5a84a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:54.637635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d1909-66d4-467d-a3d9-7197047380c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:54.638292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d1909-66d4-467d-a3d9-7197047380c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:54.638621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d1909-66d4-467d-a3d9-7197047380c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:54.638927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d1909-66d4-467d-a3d9-7197047380c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:54.652980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d1909-66d4-467d-a3d9-7197047380c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:54.654873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d1909-66d4-467d-a3d9-7197047380c5 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:54.665781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1132/2275] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:40:55.108111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.110471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:55.148964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce5e43-b72c-4b48-93c6-2300a32add6d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.185023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.185951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.186382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.186651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] PATH_INFO: `/v3/projects/fed55d34160e49fc9ae0deb6ac131c8f` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.187885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fed55d34160e49fc9ae0deb6ac131c8f)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.201436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.218289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:55.255395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fed55d34160e49fc9ae0deb6ac131c8f', 'request_id': 'req-2554bceb-9f9f-420b-a4a1-a75f35d319a9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:55.269151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fed55d34160e49fc9ae0deb6ac131c8f', 'request_id': 'req-2554bceb-9f9f-420b-a4a1-a75f35d319a9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:55.277666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fed55d34160e49fc9ae0deb6ac131c8f', 'request_id': 'req-2554bceb-9f9f-420b-a4a1-a75f35d319a9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:55.292998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] The token cache is being invalidate because project fed55d34160e49fc9ae0deb6ac131c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:40:55.293332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:55.294109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2554bceb-9f9f-420b-a4a1-a75f35d319a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:40:55.296010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1144/2276] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:40:55 2022] DELETE /identity/v3/projects/fed55d34160e49fc9ae0deb6ac131c8f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:55.302834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53da0-dc0c-4058-b09a-fd05e7169099 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.303393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53da0-dc0c-4058-b09a-fd05e7169099 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.303393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53da0-dc0c-4058-b09a-fd05e7169099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.324958 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-72b53da0-dc0c-4058-b09a-fd05e7169099 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:55.326621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-72b53da0-dc0c-4058-b09a-fd05e7169099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:55.369233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:55.393471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1133/2277] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:55.401114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8403d83-9a5a-4f82-a32d-823edf3676d6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.401403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8403d83-9a5a-4f82-a32d-823edf3676d6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.401720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8403d83-9a5a-4f82-a32d-823edf3676d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.423796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-a8403d83-9a5a-4f82-a32d-823edf3676d6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:55.425011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a8403d83-9a5a-4f82-a32d-823edf3676d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:55.433058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1145/2278] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:55.437607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53da0-dc0c-4058-b09a-fd05e7169099 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.447190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53da0-dc0c-4058-b09a-fd05e7169099 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.451900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.452488 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.452759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.453024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.453669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:40:55.453967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.455565 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.468071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1134/2279] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:40:55.472569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8403d83-9a5a-4f82-a32d-823edf3676d6 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.481768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8403d83-9a5a-4f82-a32d-823edf3676d6 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.486677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.487310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.487617 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.487896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.488516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.490438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.520379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1146/2280] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:40:55 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:55.524553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.535148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57c816-1306-4327-becb-16b066f176b2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.596827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.597274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.597551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.597831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.598481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.601028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.626750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1135/2281] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:40:55 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:40:55.628016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.640389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9545652-f43b-4eae-86e5-3d8e71bef691 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.646476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.647176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.647519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.647877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.648592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.650270 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.662121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1147/2282] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:55.668222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.679534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d4b1-231f-40be-875f-bf0d3b6078d0 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.684523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba52d26-78fb-497d-9bb6-a9f2fa9dbb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.685237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52d26-78fb-497d-9bb6-a9f2fa9dbb04 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.685676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52d26-78fb-497d-9bb6-a9f2fa9dbb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.686136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52d26-78fb-497d-9bb6-a9f2fa9dbb04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.686797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba52d26-78fb-497d-9bb6-a9f2fa9dbb04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.688452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba52d26-78fb-497d-9bb6-a9f2fa9dbb04 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.700204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1136/2283] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:40:55.705645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.717503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea4722-0943-433c-94af-b16b5f5af5d9 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.723706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27189c72-2a27-4384-9a62-6d728d3382ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.724387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27189c72-2a27-4384-9a62-6d728d3382ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.724665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27189c72-2a27-4384-9a62-6d728d3382ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.724931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27189c72-2a27-4384-9a62-6d728d3382ed admin admin] PATH_INFO: `/v3/projects/638a0c07a94f43d4807931639fd6a93e/users/f4010e99c30b4a49b19c6b8fa2b5f7f2/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.751379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27189c72-2a27-4384-9a62-6d728d3382ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=638a0c07a94f43d4807931639fd6a93e, user_id=f4010e99c30b4a49b19c6b8fa2b5f7f2, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.754036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.754327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.754493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27189c72-2a27-4384-9a62-6d728d3382ed admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.754631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.773454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1148/2284] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:40:55 2022] PUT /identity/v3/projects/638a0c07a94f43d4807931639fd6a93e/users/f4010e99c30b4a49b19c6b8fa2b5f7f2/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:40:55.776995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] MFA Rules not processed for user `bf99219df4fb45a4b7ec5fea537f8be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:55.778325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:55.778535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e1e5d-1931-475b-982e-96befea4adf1 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.778956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e1e5d-1931-475b-982e-96befea4adf1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.779328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e1e5d-1931-475b-982e-96befea4adf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.801134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c93e1e5d-1931-475b-982e-96befea4adf1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:40:55.802383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c93e1e5d-1931-475b-982e-96befea4adf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:55.847634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1137/2285] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:40:55.866813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1149/2286] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:40:55.879028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:40:55.882245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:40:55.958157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f38ce-5f5c-449d-8c24-488ce1e65c79 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:40:55.962423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:40:55.963199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:40:55.963537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:40:55.963907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:40:55.975439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:40:55.977494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:40:55.983477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:40:55.992985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1138/2287] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:41:00.763297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe32d1-b643-4c26-ad78-67be3473233d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:00.763967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe32d1-b643-4c26-ad78-67be3473233d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:00.764262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe32d1-b643-4c26-ad78-67be3473233d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:00.796691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7abe32d1-b643-4c26-ad78-67be3473233d None None] MFA Rules not processed for user `f4010e99c30b4a49b19c6b8fa2b5f7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:00.797944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7abe32d1-b643-4c26-ad78-67be3473233d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:00.836965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:00.875070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1150/2288] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:00.879768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:00.889598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629df82-40c9-4493-b119-b1561bfb0ed1 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:00.894428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:00.895196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:00.895614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:00.895994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:00.896723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:00.899227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:00.932553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1139/2289] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:41:00 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:41:00.933966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe32d1-b643-4c26-ad78-67be3473233d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:00.945872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe32d1-b643-4c26-ad78-67be3473233d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:01.010163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:01.010778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:01.011096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:01.011341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:01.012033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:01.014132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:01.041110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1151/2290] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:41:00 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:41:01.045305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:01.056044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c9fee-dd34-4cc0-9685-5a326cf99e1b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:01.061706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:01.062327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:01.062663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:01.062939 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:01.063647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:01.066119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:01.094009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1140/2291] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:41:01.097713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:01.109045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822efee4-10e4-4ce6-a0d0-9d864222242a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:01.113864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:01.114504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:01.114784 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:01.115058 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] PATH_INFO: `/v3/projects/c2b4b850f6544bc4bdeddfbce846c3ee/users/127f43b053ca42d1a1683f8a8ad5c835/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:01.135994 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b4b850f6544bc4bdeddfbce846c3ee, user_id=127f43b053ca42d1a1683f8a8ad5c835, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:01.139111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:01.154297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1152/2292] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:41:01 2022] PUT /identity/v3/projects/c2b4b850f6544bc4bdeddfbce846c3ee/users/127f43b053ca42d1a1683f8a8ad5c835/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:41:01.158435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:01.168983 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87a554-5807-4156-ac26-a725a4857433 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:01.235335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a0e21-0c4f-4ccb-ae30-2fd93b05c5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:01.236051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a0e21-0c4f-4ccb-ae30-2fd93b05c5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:01.236558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a0e21-0c4f-4ccb-ae30-2fd93b05c5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:01.236693 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a0e21-0c4f-4ccb-ae30-2fd93b05c5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:01.237251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a0e21-0c4f-4ccb-ae30-2fd93b05c5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:01.238921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a0e21-0c4f-4ccb-ae30-2fd93b05c5f3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:01.252703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1141/2293] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:41:01.256655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:01.279263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19e6e7-d6e7-4f4c-a3dd-e008cb596252 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:01.286144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:01.286946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:01.287294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:01.287673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] PATH_INFO: `/v3/projects/c2b4b850f6544bc4bdeddfbce846c3ee/users/127f43b053ca42d1a1683f8a8ad5c835/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:01.289940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b4b850f6544bc4bdeddfbce846c3ee, user_id=127f43b053ca42d1a1683f8a8ad5c835, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:01.292967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:01.305537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1153/2294] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:41:01 2022] PUT /identity/v3/projects/c2b4b850f6544bc4bdeddfbce846c3ee/users/127f43b053ca42d1a1683f8a8ad5c835/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:41:06.908518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a4866-8da2-488b-a896-8e24ef98720d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:06.909330 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a4866-8da2-488b-a896-8e24ef98720d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:06.909703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a4866-8da2-488b-a896-8e24ef98720d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:06.956108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-767a4866-8da2-488b-a896-8e24ef98720d None None] MFA Rules not processed for user `127f43b053ca42d1a1683f8a8ad5c835`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:06.957332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-767a4866-8da2-488b-a896-8e24ef98720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:07.063957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1142/2295] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:07.078963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:07.080795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:07.169133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cf17-6465-420c-85e7-4886e744eff1 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:07.176711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:07.177356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:07.177635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:07.177894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:07.205668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:07.257237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:07.258912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:07.270138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1154/2296] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:41:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:41:08.561660 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8eb34-bba0-4e1f-9fe6-1aee060ec58e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:08.562191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8eb34-bba0-4e1f-9fe6-1aee060ec58e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:08.562191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8eb34-bba0-4e1f-9fe6-1aee060ec58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:08.581817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-d5a8eb34-bba0-4e1f-9fe6-1aee060ec58e None None] MFA Rules not processed for user `f4010e99c30b4a49b19c6b8fa2b5f7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:08.583031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d5a8eb34-bba0-4e1f-9fe6-1aee060ec58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:08.608741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1143/2297] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:12.572939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:12.574655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:12.644683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ddbf2-8773-46e4-9080-ad17ed3acee0 service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:12.649506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:12.650145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:12.650468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:12.650786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] PATH_INFO: `/v3/users/a47da35ac0bf4f51a98fa3437fef1ea3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:12.652007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47da35ac0bf4f51a98fa3437fef1ea3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:12.654039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:12.699227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47da35ac0bf4f51a98fa3437fef1ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:12.699885 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47da35ac0bf4f51a98fa3437fef1ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:12.707349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47da35ac0bf4f51a98fa3437fef1ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:12.708060 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47da35ac0bf4f51a98fa3437fef1ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:12.718520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47da35ac0bf4f51a98fa3437fef1ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:12.736467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1155/2298] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:41:12 2022] DELETE /identity/v3/users/a47da35ac0bf4f51a98fa3437fef1ea3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:41:12.741056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:12.743090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:12.743386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:12.765346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:12.766579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:12.801675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:12.833380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1144/2299] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:12.843990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:12.845602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:12.930853 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:12.952874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74a238-9fe8-4391-ad5f-f1012cfa8423 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:12.957560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07943a3e-e9ec-4f0e-be46-597bcc2599d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:12.959785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07943a3e-e9ec-4f0e-be46-597bcc2599d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:12.960593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07943a3e-e9ec-4f0e-be46-597bcc2599d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:12.961391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07943a3e-e9ec-4f0e-be46-597bcc2599d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:12.977205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07943a3e-e9ec-4f0e-be46-597bcc2599d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:12.978953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07943a3e-e9ec-4f0e-be46-597bcc2599d2 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:13.003609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1156/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:41:13.766324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:13.768105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:13.797498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5a4f4-678d-4dc0-b1bf-02191531b429 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:13.804490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:13.805418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:13.805864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:13.806280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] PATH_INFO: `/v3/projects/cc53e85e05b84a4790f069f13ab09743` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:13.807618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc53e85e05b84a4790f069f13ab09743)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:13.810170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:13.875284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc53e85e05b84a4790f069f13ab09743', 'request_id': 'req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:13.881036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc53e85e05b84a4790f069f13ab09743', 'request_id': 'req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:13.887772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc53e85e05b84a4790f069f13ab09743', 'request_id': 'req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:13.907989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] The token cache is being invalidate because project cc53e85e05b84a4790f069f13ab09743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:41:13.908354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:13.909093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9dc535b4-4ec0-4856-8bce-d62da5a8abaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:13.910754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1145/2301] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:41:13 2022] DELETE /identity/v3/projects/cc53e85e05b84a4790f069f13ab09743 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:41:13.915754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd693a-9bf4-4f2f-9fbb-01c8fd2951f1 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:13.916258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd693a-9bf4-4f2f-9fbb-01c8fd2951f1 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:13.916353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd693a-9bf4-4f2f-9fbb-01c8fd2951f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:13.936806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b6bd693a-9bf4-4f2f-9fbb-01c8fd2951f1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:13.938053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b6bd693a-9bf4-4f2f-9fbb-01c8fd2951f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:13.998662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1157/2302] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:14.006783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b81f5-a5ac-4280-aa33-68121f599832 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.007074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b81f5-a5ac-4280-aa33-68121f599832 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.007346 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b81f5-a5ac-4280-aa33-68121f599832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.029410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-064b81f5-a5ac-4280-aa33-68121f599832 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:14.030624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-064b81f5-a5ac-4280-aa33-68121f599832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:14.039052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1146/2303] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:14.043240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd693a-9bf4-4f2f-9fbb-01c8fd2951f1 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:14.053416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd693a-9bf4-4f2f-9fbb-01c8fd2951f1 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:14.058410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:14.059031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.059310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.059613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.060401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:41:14.060683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:14.062336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:14.074359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1158/2304] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:41:14.078758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b81f5-a5ac-4280-aa33-68121f599832 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:14.087916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b81f5-a5ac-4280-aa33-68121f599832 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:14.093214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:14.093808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.094059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.094345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.094972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:14.096868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:14.126232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1147/2305] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:41:14 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:41:14.130625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:14.141115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc701905-ce29-4d7c-8eb0-b53050dd8940 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:14.196807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:14.197416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.197682 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.197945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.198555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:14.200573 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:14.225725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1159/2306] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:41:14 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:41:14.231556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:14.242462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470064c-bb95-46a0-98fb-2dfc1fc4f778 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:14.248492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:14.249092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.249364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.249631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.250271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:14.251908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:14.262219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1148/2307] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:41:14.266814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:14.277869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76614c-6259-4494-8d17-876a2fe2f4d2 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:14.283647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d06c2a-c61b-4d45-b5aa-519fb7e52356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:14.284272 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06c2a-c61b-4d45-b5aa-519fb7e52356 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.284551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06c2a-c61b-4d45-b5aa-519fb7e52356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.284821 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06c2a-c61b-4d45-b5aa-519fb7e52356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.285468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d06c2a-c61b-4d45-b5aa-519fb7e52356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:14.287074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d06c2a-c61b-4d45-b5aa-519fb7e52356 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:14.298439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1160/2308] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:41:14.302712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:14.313180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed78f-79ef-4f0f-b124-df6d52c0c360 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:14.318951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:14.319691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.319976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.320251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] PATH_INFO: `/v3/projects/e0ba1bb080a9456a9a5b567f33c45bd3/users/81d09306a85048e19c8b83b787d81877/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.336014 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ba1bb080a9456a9a5b567f33c45bd3, user_id=81d09306a85048e19c8b83b787d81877, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:14.338686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:14.360433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1149/2309] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:41:14 2022] PUT /identity/v3/projects/e0ba1bb080a9456a9a5b567f33c45bd3/users/81d09306a85048e19c8b83b787d81877/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:41:14.367611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d25da2-8f97-4d91-86dc-e44a018258bd None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.367917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d25da2-8f97-4d91-86dc-e44a018258bd None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.368305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d25da2-8f97-4d91-86dc-e44a018258bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.408728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-06d25da2-8f97-4d91-86dc-e44a018258bd None None] MFA Rules not processed for user `81d09306a85048e19c8b83b787d81877`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:14.409982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-06d25da2-8f97-4d91-86dc-e44a018258bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:14.452132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:14.480097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1161/2310] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:14.491307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:14.493077 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:14.567356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf6c6-9b90-440b-b628-55b4212e0ea2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:14.571420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66680b9a-0524-42a6-a55e-e0b958e598c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:14.572220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680b9a-0524-42a6-a55e-e0b958e598c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:14.572494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680b9a-0524-42a6-a55e-e0b958e598c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:14.572767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680b9a-0524-42a6-a55e-e0b958e598c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:14.581731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66680b9a-0524-42a6-a55e-e0b958e598c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:14.583244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66680b9a-0524-42a6-a55e-e0b958e598c3 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:14.592582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1150/2311] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:41:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:41:21.262468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46607364-ba0e-44a5-a9d3-f2fb4972d576 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:21.263001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46607364-ba0e-44a5-a9d3-f2fb4972d576 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:21.263094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46607364-ba0e-44a5-a9d3-f2fb4972d576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:21.284336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-46607364-ba0e-44a5-a9d3-f2fb4972d576 None None] MFA Rules not processed for user `81d09306a85048e19c8b83b787d81877`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:21.285561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-46607364-ba0e-44a5-a9d3-f2fb4972d576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:21.293490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1162/2312] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:22.556610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477b6da-be98-4e50-8a89-47ad5d519689 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:22.557825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477b6da-be98-4e50-8a89-47ad5d519689 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:22.558761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477b6da-be98-4e50-8a89-47ad5d519689 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:22.560914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1151/2313] 10.222.0.18 () {54 vars in 961 bytes} [Mon Aug 22 05:41:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:41:22.566927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:22.567478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:22.567478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:22.587407 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:22.588683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:22.656415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1163/2314] 10.222.0.18 () {58 vars in 1058 bytes} [Mon Aug 22 05:41:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:22.664141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477b6da-be98-4e50-8a89-47ad5d519689 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:22.672954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:22.682049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477b6da-be98-4e50-8a89-47ad5d519689 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:22.687923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646044-e921-45cc-b90f-884de90707d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:22.688580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646044-e921-45cc-b90f-884de90707d5 service placement] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:22.688947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646044-e921-45cc-b90f-884de90707d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:22.689251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646044-e921-45cc-b90f-884de90707d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:22.691256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-40646044-e921-45cc-b90f-884de90707d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:22.702933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646044-e921-45cc-b90f-884de90707d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:22.704586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646044-e921-45cc-b90f-884de90707d5 service placement] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:22.714269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1152/2315] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:41:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:41:25.471538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:25.473119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:25.612359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc0a20-dfee-4f0d-9db7-c3b491043217 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:25.617849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:25.618506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:25.618789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:25.619067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] PATH_INFO: `/v3/users/e098fd4d56704a6cb0ba1d443fc41faa` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:25.620380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e098fd4d56704a6cb0ba1d443fc41faa)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:25.622623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:25.696472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e098fd4d56704a6cb0ba1d443fc41faa'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:25.697787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e098fd4d56704a6cb0ba1d443fc41faa'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:25.708263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e098fd4d56704a6cb0ba1d443fc41faa'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:25.709523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e098fd4d56704a6cb0ba1d443fc41faa'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:25.726946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e098fd4d56704a6cb0ba1d443fc41faa'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:25.741129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:25.767789 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1164/2316] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:41:25 2022] DELETE /identity/v3/users/e098fd4d56704a6cb0ba1d443fc41faa => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:41:25.771202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89be7d-822a-4e30-ade5-938da1e23bc5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:25.772262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89be7d-822a-4e30-ade5-938da1e23bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:25.772729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89be7d-822a-4e30-ade5-938da1e23bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:25.809170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-cf89be7d-822a-4e30-ade5-938da1e23bc5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:25.811554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cf89be7d-822a-4e30-ade5-938da1e23bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:25.940039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1153/2317] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:25.964842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:25.967185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:26.087300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8798a7-f709-44af-94f3-b500aa3c2ab5 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:26.100637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:26.102315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:26.102990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:26.103502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:26.120134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:26.121795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:26.135778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1165/2318] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:41:26.141979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e9a57-9515-444c-8065-7c441477b329 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:26.144473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e9a57-9515-444c-8065-7c441477b329 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:26.144766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e9a57-9515-444c-8065-7c441477b329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:26.163745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-876e9a57-9515-444c-8065-7c441477b329 None None] MFA Rules not processed for user `9643b0f10e8345889c181efbb1978237`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:26.164932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-876e9a57-9515-444c-8065-7c441477b329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:26.234015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1154/2319] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:26.599268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:26.601494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:26.621663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d901e-cbd3-4363-b2c5-99f737023b8a service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:26.628715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:26.629790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:26.630388 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:26.630972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] PATH_INFO: `/v3/projects/110fa73293ea478e8fdc8819c9809cd8` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:26.633158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=110fa73293ea478e8fdc8819c9809cd8)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:26.636227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:26.693208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110fa73293ea478e8fdc8819c9809cd8', 'request_id': 'req-3a2cdd89-9305-4132-a970-3bacd232cb46'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:26.699656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110fa73293ea478e8fdc8819c9809cd8', 'request_id': 'req-3a2cdd89-9305-4132-a970-3bacd232cb46'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:26.706257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110fa73293ea478e8fdc8819c9809cd8', 'request_id': 'req-3a2cdd89-9305-4132-a970-3bacd232cb46'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:26.719891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] The token cache is being invalidate because project 110fa73293ea478e8fdc8819c9809cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:41:26.719891 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:26.720452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2cdd89-9305-4132-a970-3bacd232cb46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:26.722039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1166/2320] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:41:26 2022] DELETE /identity/v3/projects/110fa73293ea478e8fdc8819c9809cd8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:41:26.726950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2b71a-70bc-4ee2-bc63-1baecc98951d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:26.727318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2b71a-70bc-4ee2-bc63-1baecc98951d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:26.727616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2b71a-70bc-4ee2-bc63-1baecc98951d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:26.750114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-aaf2b71a-70bc-4ee2-bc63-1baecc98951d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:26.751335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aaf2b71a-70bc-4ee2-bc63-1baecc98951d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:26.813194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1155/2321] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:26.821142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe700a-bde9-4e07-8b1e-7716adb14ff2 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:26.821504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe700a-bde9-4e07-8b1e-7716adb14ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:26.821785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe700a-bde9-4e07-8b1e-7716adb14ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:26.842643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-09fe700a-bde9-4e07-8b1e-7716adb14ff2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:26.844520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-09fe700a-bde9-4e07-8b1e-7716adb14ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:26.852799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1167/2322] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:26.857023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2b71a-70bc-4ee2-bc63-1baecc98951d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:26.867370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2b71a-70bc-4ee2-bc63-1baecc98951d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:26.871988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:26.872595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:26.872873 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:26.873144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:26.873791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:41:26.874092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:26.876410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:26.891345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1156/2323] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:41:26.892832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe700a-bde9-4e07-8b1e-7716adb14ff2 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:26.902922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe700a-bde9-4e07-8b1e-7716adb14ff2 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:26.909141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:26.909912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:26.910261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:26.910624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:26.911389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:26.913383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:26.944822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1168/2324] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:41:26 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:41:26.949041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:26.960187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323692eb-5e56-4d6c-823b-c77027fbdf0b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:26.985898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:27.043719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:27.044579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:27.045006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:27.045424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:27.046240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:27.049211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:27.075898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1157/2325] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:41:26 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:41:27.080230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:27.096364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7c72-7f3f-4267-86a6-355d15954dce admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:27.101949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:27.102870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:27.103217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:27.103568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:27.104594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:27.106569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:27.125021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1169/2326] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:41:27.130602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:27.146357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a16865-02af-4c26-b6d8-8f8a87155d17 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:27.170425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83222879-a21a-456e-9d67-243649a50531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:27.173638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83222879-a21a-456e-9d67-243649a50531 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:27.173768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83222879-a21a-456e-9d67-243649a50531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:27.175078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83222879-a21a-456e-9d67-243649a50531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:27.178709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83222879-a21a-456e-9d67-243649a50531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:27.181957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83222879-a21a-456e-9d67-243649a50531 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:27.229738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1158/2327] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:41:27.233766 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:27.244616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463b027-8330-4e5e-b089-ce8eb4848039 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:27.251990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:27.252759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:27.253112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:27.253491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] PATH_INFO: `/v3/projects/43aad522eb7a49e095822ac26cb318de/users/f544214fa7994c69a38a8b5d5dd1c2cb/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:27.275281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43aad522eb7a49e095822ac26cb318de, user_id=f544214fa7994c69a38a8b5d5dd1c2cb, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:27.279140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:27.299066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1170/2328] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:41:27 2022] PUT /identity/v3/projects/43aad522eb7a49e095822ac26cb318de/users/f544214fa7994c69a38a8b5d5dd1c2cb/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:41:27.313970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651c0fe-3319-466c-8d79-c9e6c818cbc3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:27.314252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651c0fe-3319-466c-8d79-c9e6c818cbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:27.314613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651c0fe-3319-466c-8d79-c9e6c818cbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:27.344034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-d651c0fe-3319-466c-8d79-c9e6c818cbc3 None None] MFA Rules not processed for user `f544214fa7994c69a38a8b5d5dd1c2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:27.345332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d651c0fe-3319-466c-8d79-c9e6c818cbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:27.426498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1159/2329] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:27.435827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:27.437808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:27.453911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:27.538768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef81ed9-6fde-44d0-9aa9-dab8ee17c561 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:27.543809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:27.544686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:27.545117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:27.545694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:27.561750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:27.563752 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:27.574576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1171/2330] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:41:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:41:37.418469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:37.569548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=21 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:38.295275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0410ff-2c7a-4cdb-a465-c0cc62a12e0a None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:38.295717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0410ff-2c7a-4cdb-a465-c0cc62a12e0a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:38.296089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0410ff-2c7a-4cdb-a465-c0cc62a12e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:38.318476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6c0410ff-2c7a-4cdb-a465-c0cc62a12e0a None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:38.320159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6c0410ff-2c7a-4cdb-a465-c0cc62a12e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:38.403612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1160/2331] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:41:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:38.452088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:38.453567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:38.539138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d83d3-d753-49f9-9fd4-4913cb1d914b service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:38.545320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:38.546065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:38.546409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:38.546788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:38.558438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:38.560837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:38.574030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1172/2332] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:41:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:41:48.392750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=25 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:48.570139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=32 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:56.282393 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00961723-245b-415e-b84e-1ddfc7adea78 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:56.283220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00961723-245b-415e-b84e-1ddfc7adea78 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:56.283805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00961723-245b-415e-b84e-1ddfc7adea78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:56.306047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-00961723-245b-415e-b84e-1ddfc7adea78 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:56.307314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-00961723-245b-415e-b84e-1ddfc7adea78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:56.314459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1161/2333] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 22 05:41:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:41:56.328158 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:56.338854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c00bec-28d6-4715-a21e-8c186e8ce053 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:56.344775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:56.345396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:56.345726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:56.346015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:56.354613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:56.356342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:56.365064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1173/2334] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:41:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:41:57.149396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fe7bd-1005-4e66-b998-e5847fe1d281 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:57.149683 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fe7bd-1005-4e66-b998-e5847fe1d281 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:57.149984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fe7bd-1005-4e66-b998-e5847fe1d281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:57.173061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-ea7fe7bd-1005-4e66-b998-e5847fe1d281 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:57.174368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ea7fe7bd-1005-4e66-b998-e5847fe1d281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:57.185005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1162/2335] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:41:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:57.197366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:57.208070 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950471a-e520-4386-97de-8c144cb8abd4 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:57.213211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:57.222401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:57.222688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:57.222963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:57.233650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:57.235239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:57.244815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1174/2336] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:41:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:41:57.790549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af404849-d5b8-4385-bc59-a5d0070d4590 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:57.791046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af404849-d5b8-4385-bc59-a5d0070d4590 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:57.791150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af404849-d5b8-4385-bc59-a5d0070d4590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:57.949449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-af404849-d5b8-4385-bc59-a5d0070d4590 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:57.951208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-af404849-d5b8-4385-bc59-a5d0070d4590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:57.960651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1163/2337] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 22 05:41:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:41:57.980583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:57.993041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532773b-17e9-4136-9787-82e23c206b34 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:57.998607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:57.999237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:57.999730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:58.000168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:58.009723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:58.011340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:58.020340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1175/2338] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:41:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:41:58.360855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187275e-130e-41e2-a260-311fec751664 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:58.361273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187275e-130e-41e2-a260-311fec751664 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:58.362064 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187275e-130e-41e2-a260-311fec751664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:58.382148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-d187275e-130e-41e2-a260-311fec751664 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:58.383586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d187275e-130e-41e2-a260-311fec751664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:58.391788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1164/2339] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:41:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:58.407459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:58.420216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468b5e5-e205-4c0d-9291-960dfe2a6b49 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:58.429159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b095417-2107-4702-be52-ef36666efc57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:58.429901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095417-2107-4702-be52-ef36666efc57 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:58.430263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095417-2107-4702-be52-ef36666efc57 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:58.430909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095417-2107-4702-be52-ef36666efc57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:58.440798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b095417-2107-4702-be52-ef36666efc57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:58.442816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b095417-2107-4702-be52-ef36666efc57 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:58.452716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1176/2340] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:41:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:41:58.620996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b733f-80f7-4f56-91da-e23f78fca73a None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:58.621524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b733f-80f7-4f56-91da-e23f78fca73a None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:58.621792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b733f-80f7-4f56-91da-e23f78fca73a None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:58.623133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1165/2341] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:41:58.634517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:58.634988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:58.635360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:58.668386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] MFA Rules not processed for user `733035fd9ee6414dbc23c69bb9206b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:58.669613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:58.715020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b733f-80f7-4f56-91da-e23f78fca73a None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:58.717694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-573b733f-80f7-4f56-91da-e23f78fca73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:58.827788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1177/2342] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:41:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:41:58.840487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:58.842455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:58.851703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b733f-80f7-4f56-91da-e23f78fca73a None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:58.857513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:58.858582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:58.859040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:58.859502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] PATH_INFO: `/v3/users/bf99219df4fb45a4b7ec5fea537f8be2` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:58.860868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf99219df4fb45a4b7ec5fea537f8be2)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:58.864161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:58.868222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a325f-3a71-4444-bc29-5bb7579aea43 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:58.884171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfae26-69e3-4f40-b1d7-dca5347b1ea1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:58.884867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfae26-69e3-4f40-b1d7-dca5347b1ea1 service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:58.885146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfae26-69e3-4f40-b1d7-dca5347b1ea1 service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:58.886307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfae26-69e3-4f40-b1d7-dca5347b1ea1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:58.907658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfae26-69e3-4f40-b1d7-dca5347b1ea1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:58.910108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfae26-69e3-4f40-b1d7-dca5347b1ea1 service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:58.922658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1178/2343] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:41:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:41:58.949127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf99219df4fb45a4b7ec5fea537f8be2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:58.950244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf99219df4fb45a4b7ec5fea537f8be2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:58.951476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf99219df4fb45a4b7ec5fea537f8be2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:58.968938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf99219df4fb45a4b7ec5fea537f8be2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:58.984254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf99219df4fb45a4b7ec5fea537f8be2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:59.005550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.006077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.006175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.008899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1166/2344] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:41:58 2022] DELETE /identity/v3/users/bf99219df4fb45a4b7ec5fea537f8be2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:41:59.042607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:59.044065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:59.115395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1179/2345] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 22 05:41:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:41:59.130173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:59.131625 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:59.195407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:59.207066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:59.208613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:59.226589 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a463f9-9d01-4024-8491-a6d5b0fd278c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:59.233215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce42f66-f690-4cf6-970a-7c718d126026 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:59.233884 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce42f66-f690-4cf6-970a-7c718d126026 service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.234158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce42f66-f690-4cf6-970a-7c718d126026 service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.234490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce42f66-f690-4cf6-970a-7c718d126026 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.245327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce42f66-f690-4cf6-970a-7c718d126026 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:59.247067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce42f66-f690-4cf6-970a-7c718d126026 service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:59.271794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1167/2346] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:41:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:41:59.291505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:41:59.323065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b37537-8776-4334-903e-9a5dff6def09 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:59.328283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:59.329055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.329342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.329614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] PATH_INFO: `/v3/projects/375a7cf8cbc042a187f2b144d907c175` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.330806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375a7cf8cbc042a187f2b144d907c175)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:59.332801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:59.425291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375a7cf8cbc042a187f2b144d907c175', 'request_id': 'req-72f3979b-b347-4493-82df-f0e982e35929'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:59.442568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375a7cf8cbc042a187f2b144d907c175', 'request_id': 'req-72f3979b-b347-4493-82df-f0e982e35929'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:59.448511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375a7cf8cbc042a187f2b144d907c175', 'request_id': 'req-72f3979b-b347-4493-82df-f0e982e35929'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:59.473405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] The token cache is being invalidate because project 375a7cf8cbc042a187f2b144d907c175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:41:59.473818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:59.474711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-72f3979b-b347-4493-82df-f0e982e35929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:41:59.476775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1180/2347] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:41:59 2022] DELETE /identity/v3/projects/375a7cf8cbc042a187f2b144d907c175 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:41:59.482552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813595a-8fe8-41ab-81ab-425c1a5065e8 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.482848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813595a-8fe8-41ab-81ab-425c1a5065e8 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.483452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813595a-8fe8-41ab-81ab-425c1a5065e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.510580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6813595a-8fe8-41ab-81ab-425c1a5065e8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:59.511840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6813595a-8fe8-41ab-81ab-425c1a5065e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:59.622074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1168/2348] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:41:59.623803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d9cea-ff5b-4586-b775-8c1e5526231b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.624089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d9cea-ff5b-4586-b775-8c1e5526231b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.624358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d9cea-ff5b-4586-b775-8c1e5526231b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.650220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f13d9cea-ff5b-4586-b775-8c1e5526231b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:41:59.651558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f13d9cea-ff5b-4586-b775-8c1e5526231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:41:59.678881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1181/2349] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:41:59.683484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813595a-8fe8-41ab-81ab-425c1a5065e8 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:59.696135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813595a-8fe8-41ab-81ab-425c1a5065e8 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:59.701590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:59.702335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.702667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.702996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.703786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:41:59.704154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:59.706040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:59.722318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1169/2350] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:41:59.726897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d9cea-ff5b-4586-b775-8c1e5526231b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:59.737170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d9cea-ff5b-4586-b775-8c1e5526231b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:59.744122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:59.744777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.745050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.745319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.745968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:59.748020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:59.791176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1182/2351] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:41:59 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:41:59.792406 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:59.804197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705921d-b571-47ff-8551-4b30679504f8 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:59.882668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:59.884302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.884302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.884302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.884508 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:59.886881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:59.927664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1170/2352] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:41:59 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:41:59.935990 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:59.949154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd997b5-eb39-47cf-84fc-c81997a5bf9b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:59.953989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:59.954604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.954882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.955152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.956146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:59.957910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:41:59.968613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1183/2353] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:41:59.973331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:41:59.983420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c493f3-6352-487e-8d53-e175b5506781 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:41:59.988328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95948788-8321-45b3-87a8-174a691213a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:41:59.988879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95948788-8321-45b3-87a8-174a691213a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:41:59.989154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95948788-8321-45b3-87a8-174a691213a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:41:59.989419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95948788-8321-45b3-87a8-174a691213a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:41:59.990058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95948788-8321-45b3-87a8-174a691213a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:41:59.991676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95948788-8321-45b3-87a8-174a691213a3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:00.002946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1171/2354] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:42:00.007441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:00.018902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b3a0-e9cb-4ecd-977d-db48c1670e32 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:00.024146 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:00.024844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:00.025125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:00.025395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] PATH_INFO: `/v3/projects/cd1437d403974f5398de8342a65f9061/users/2d2864594bb34aecb636864b07f84bc8/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:00.042330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1437d403974f5398de8342a65f9061, user_id=2d2864594bb34aecb636864b07f84bc8, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:00.045075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:00.059683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1184/2355] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:42:00 2022] PUT /identity/v3/projects/cd1437d403974f5398de8342a65f9061/users/2d2864594bb34aecb636864b07f84bc8/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:42:00.071163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:00.071447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:00.071803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:00.100569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] MFA Rules not processed for user `2d2864594bb34aecb636864b07f84bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:42:00.101780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:00.152482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:42:00.180063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1172/2356] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:42:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:42:00.192547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:00.194340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:00.264559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd227db-6c99-4c23-a4ff-e62222b07a6c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:00.268465 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4173579-993d-4d7c-b3a1-84b4ecba1eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:00.269234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173579-993d-4d7c-b3a1-84b4ecba1eab service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:00.269536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173579-993d-4d7c-b3a1-84b4ecba1eab service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:00.269758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173579-993d-4d7c-b3a1-84b4ecba1eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:00.278499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4173579-993d-4d7c-b3a1-84b4ecba1eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:00.279981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4173579-993d-4d7c-b3a1-84b4ecba1eab service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:00.289471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1185/2357] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:42:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:42:01.667124 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:01.668651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:01.742567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379334f6-dea1-400a-bfb5-29f7075bf9ad None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:01.746984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:01.747673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:01.747944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:01.748208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] PATH_INFO: `/v3/users/81d09306a85048e19c8b83b787d81877` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:01.749325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d09306a85048e19c8b83b787d81877)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:01.751212 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:01.799821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d09306a85048e19c8b83b787d81877'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:01.800723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d09306a85048e19c8b83b787d81877'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:01.801348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d09306a85048e19c8b83b787d81877'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:01.809418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d09306a85048e19c8b83b787d81877'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:01.817377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d09306a85048e19c8b83b787d81877'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:01.840327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1173/2358] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:42:01 2022] DELETE /identity/v3/users/81d09306a85048e19c8b83b787d81877 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:42:01.845967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:01.846566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:01.846566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:01.858302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:42:01.868877 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:42:01.870926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:01.938527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1186/2359] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:42:01.949280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:01.950923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:02.020278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0766-d4dd-47f0-9967-245f2eb272ff admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:02.025803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:02.026418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:02.026693 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:02.026993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:02.036563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:02.038321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:02.043259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:42:02.048196 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1174/2360] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:42:02.222100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:02.223494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:02.241064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1e52-3ac3-4f09-b31d-99993251b46a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:02.246794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:02.247461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:02.247748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:02.248029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] PATH_INFO: `/v3/projects/e0ba1bb080a9456a9a5b567f33c45bd3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:02.249410 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ba1bb080a9456a9a5b567f33c45bd3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:02.251659 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:02.303628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ba1bb080a9456a9a5b567f33c45bd3', 'request_id': 'req-9fcb368a-d320-4884-93aa-991792779729'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:02.309029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ba1bb080a9456a9a5b567f33c45bd3', 'request_id': 'req-9fcb368a-d320-4884-93aa-991792779729'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:02.315078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ba1bb080a9456a9a5b567f33c45bd3', 'request_id': 'req-9fcb368a-d320-4884-93aa-991792779729'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:02.329555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] The token cache is being invalidate because project e0ba1bb080a9456a9a5b567f33c45bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:42:02.329840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:02.330452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:02.331953 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1187/2361] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:42:02 2022] DELETE /identity/v3/projects/e0ba1bb080a9456a9a5b567f33c45bd3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:42:12.326786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:42:28.319560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:28.328626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176837-eef9-46f1-9be1-06267bdc3e7c service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:28.385765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2195ed-b7a8-4209-a854-2a8e1cae5345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:28.386595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2195ed-b7a8-4209-a854-2a8e1cae5345 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:28.387131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2195ed-b7a8-4209-a854-2a8e1cae5345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:28.387500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2195ed-b7a8-4209-a854-2a8e1cae5345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:28.389448 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ab2195ed-b7a8-4209-a854-2a8e1cae5345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:28.460234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2195ed-b7a8-4209-a854-2a8e1cae5345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:28.461670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2195ed-b7a8-4209-a854-2a8e1cae5345 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:28.472793 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1175/2362] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:42:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:42:31.148749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:31.150504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:31.230010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb368a-d320-4884-93aa-991792779729 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:31.235667 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:31.236329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:31.236656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:31.236973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] PATH_INFO: `/v3/users/2d2864594bb34aecb636864b07f84bc8` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:31.238220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2864594bb34aecb636864b07f84bc8)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:31.240304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:31.292764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2864594bb34aecb636864b07f84bc8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.293515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2864594bb34aecb636864b07f84bc8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.301538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2864594bb34aecb636864b07f84bc8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.302308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2864594bb34aecb636864b07f84bc8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.311568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2864594bb34aecb636864b07f84bc8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.330948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1188/2363] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:42:31 2022] DELETE /identity/v3/users/2d2864594bb34aecb636864b07f84bc8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:42:31.335291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:31.335853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:31.335950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:31.362088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:42:31.363321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:31.437896 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1176/2364] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:42:31.447462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:31.449248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:31.516061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2ca1a-eccc-430b-8ed3-1ac309e06f0c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:31.519753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:31.520378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:31.520657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:31.520931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:31.530954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:31.532450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:31.542665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1189/2365] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:42:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:42:31.829459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:31.831033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:31.843880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42d199-7f1f-4e64-b74d-8f4fd37f41e2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:31.849394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:31.850231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:31.850698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:31.851273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] PATH_INFO: `/v3/projects/cd1437d403974f5398de8342a65f9061` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:31.852761 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1437d403974f5398de8342a65f9061)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:31.854818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:31.913647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:42:31.916473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1437d403974f5398de8342a65f9061', 'request_id': 'req-09193c5a-ed69-49d7-9baa-ed3af56062fe'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.928204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1437d403974f5398de8342a65f9061', 'request_id': 'req-09193c5a-ed69-49d7-9baa-ed3af56062fe'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.935121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1437d403974f5398de8342a65f9061', 'request_id': 'req-09193c5a-ed69-49d7-9baa-ed3af56062fe'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.951395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] The token cache is being invalidate because project cd1437d403974f5398de8342a65f9061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:42:31.951759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.952378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-09193c5a-ed69-49d7-9baa-ed3af56062fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:31.954682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1177/2366] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:42:31 2022] DELETE /identity/v3/projects/cd1437d403974f5398de8342a65f9061 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:42:37.381863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:37.383558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:37.397776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237afc63-95c3-4af3-8d44-e97e676cf697 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:37.424271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:42:37.469673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:37.470314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:37.470590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:37.470839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] PATH_INFO: `/v3/users/f544214fa7994c69a38a8b5d5dd1c2cb` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:37.472120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f544214fa7994c69a38a8b5d5dd1c2cb)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:37.474100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:37.521707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f544214fa7994c69a38a8b5d5dd1c2cb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:37.522346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f544214fa7994c69a38a8b5d5dd1c2cb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:37.529263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f544214fa7994c69a38a8b5d5dd1c2cb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:37.529924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f544214fa7994c69a38a8b5d5dd1c2cb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:37.538086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f544214fa7994c69a38a8b5d5dd1c2cb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:37.555441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1190/2367] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:42:37 2022] DELETE /identity/v3/users/f544214fa7994c69a38a8b5d5dd1c2cb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:42:37.560022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:37.560502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:37.560631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:37.581496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:42:37.582698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:37.661399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1178/2368] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:42:37.671875 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:37.673543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:37.747117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a8add-b33f-48aa-b793-ce8e54451b68 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:37.752120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:37.752725 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:37.753012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:37.753354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:37.762897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:37.764505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:37.774558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1191/2369] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:42:38.075270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:42:38.076846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:42:38.090532 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646472f-c017-4618-b182-2b2b5956a3d9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:42:38.097232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:42:38.097935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:42:38.098211 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:42:38.098481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] PATH_INFO: `/v3/projects/43aad522eb7a49e095822ac26cb318de` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:42:38.099762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43aad522eb7a49e095822ac26cb318de)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:42:38.106171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:42:38.157382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43aad522eb7a49e095822ac26cb318de', 'request_id': 'req-3cbd161f-ab54-4861-8ac2-11276dd41f6f'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:38.163243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43aad522eb7a49e095822ac26cb318de', 'request_id': 'req-3cbd161f-ab54-4861-8ac2-11276dd41f6f'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:38.169569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43aad522eb7a49e095822ac26cb318de', 'request_id': 'req-3cbd161f-ab54-4861-8ac2-11276dd41f6f'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:38.184176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] The token cache is being invalidate because project 43aad522eb7a49e095822ac26cb318de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:42:38.184446 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:38.185091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:42:38.186618 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1179/2370] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:42:38 2022] DELETE /identity/v3/projects/43aad522eb7a49e095822ac26cb318de => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:42:47.770165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:42:48.181294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:DELETE=11,keystone:SELECT=61,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:43:12.541213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:12.542761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:12.610430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed4348-d924-4ab9-8cb5-0a81e7d4a769 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:12.671992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc92c7-456d-45c1-bcaf-7487cf115ddf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:12.672508 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc92c7-456d-45c1-bcaf-7487cf115ddf service glance] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:12.672790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc92c7-456d-45c1-bcaf-7487cf115ddf service glance] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:12.673061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc92c7-456d-45c1-bcaf-7487cf115ddf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:12.674945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bebc92c7-456d-45c1-bcaf-7487cf115ddf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:12.752471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc92c7-456d-45c1-bcaf-7487cf115ddf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:12.753933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc92c7-456d-45c1-bcaf-7487cf115ddf service glance] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:12.765253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1192/2371] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:12.942697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:12.944215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:13.005907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd161f-ab54-4861-8ac2-11276dd41f6f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:13.009882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:13.010501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:13.010776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:13.011046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:13.012868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:13.026731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:13.028241 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:13.039491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1180/2372] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:22.760329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=74 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:43:23.034688 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=26 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:43:36.025363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:36.025862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:36.025988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:36.045581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:43:36.046791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:36.103023 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1193/2373] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:43:36.156804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:36.158436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:36.239745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb701f-6532-4684-9217-81de356f3b49 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:36.246301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:36.246945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:36.247250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:36.247514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:36.256429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:36.258016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:36.267267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1181/2374] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:43:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:43:41.850930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:41.853343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:41.873740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c1446-837c-49ab-884f-26905a1ae493 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:41.881865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:41.882490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:41.883128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:41.884264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:41.891018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:41.975195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:41.977145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:41.987840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1194/2375] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:46.263669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=32 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:43:47.001549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:47.011821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe11b73-b118-480b-9606-46868a5b3299 service swift] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:47.016678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c47487-f58e-4936-859b-561cbcdbef5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:47.017333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47487-f58e-4936-859b-561cbcdbef5c service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:47.017617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47487-f58e-4936-859b-561cbcdbef5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:47.017880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47487-f58e-4936-859b-561cbcdbef5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:47.019946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-73c47487-f58e-4936-859b-561cbcdbef5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:47.030219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c47487-f58e-4936-859b-561cbcdbef5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:47.031687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c47487-f58e-4936-859b-561cbcdbef5c service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:47.039887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1182/2376] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:51.130358 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:51.140576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdb70a-80ed-4b77-bf67-552e2b7186d5 service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:51.144427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d381a-b9d7-43a0-a086-298f3b296550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:51.145044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d381a-b9d7-43a0-a086-298f3b296550 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.145327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d381a-b9d7-43a0-a086-298f3b296550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.145598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d381a-b9d7-43a0-a086-298f3b296550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.147763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8e9d381a-b9d7-43a0-a086-298f3b296550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:51.158608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d381a-b9d7-43a0-a086-298f3b296550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:51.160325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d381a-b9d7-43a0-a086-298f3b296550 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:51.169221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1195/2377] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:51.316198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd14b-3f4e-4626-a5d9-53f33ad2c8db None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.316751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd14b-3f4e-4626-a5d9-53f33ad2c8db None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.316834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcd14b-3f4e-4626-a5d9-53f33ad2c8db None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.318243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1183/2378] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:43:51.336486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fddbcf-6144-4309-9a73-8ee433f6c100 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.336775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fddbcf-6144-4309-9a73-8ee433f6c100 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.337054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fddbcf-6144-4309-9a73-8ee433f6c100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.357284 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-59fddbcf-6144-4309-9a73-8ee433f6c100 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:43:51.358503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-59fddbcf-6144-4309-9a73-8ee433f6c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:51.369364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1196/2379] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:43:51.381472 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcd14b-3f4e-4626-a5d9-53f33ad2c8db None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:51.383232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4bbcd14b-3f4e-4626-a5d9-53f33ad2c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:51.398531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcd14b-3f4e-4626-a5d9-53f33ad2c8db None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:51.402777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7ef89-9cd9-4ad7-bafb-873a24212f81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:51.403633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7ef89-9cd9-4ad7-bafb-873a24212f81 service glance] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.404202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7ef89-9cd9-4ad7-bafb-873a24212f81 service glance] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.404604 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7ef89-9cd9-4ad7-bafb-873a24212f81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.418641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7ef89-9cd9-4ad7-bafb-873a24212f81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:51.420436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7ef89-9cd9-4ad7-bafb-873a24212f81 service glance] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:51.431988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1184/2380] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:43:51.508327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a4f5e-ee29-4d00-8f89-7f76ca51e1c0 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.508655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a4f5e-ee29-4d00-8f89-7f76ca51e1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.508932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a4f5e-ee29-4d00-8f89-7f76ca51e1c0 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.510275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1197/2381] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:43:51.515926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f840ad2-d3a2-4a42-a9f0-04a74188b620 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.516209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f840ad2-d3a2-4a42-a9f0-04a74188b620 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.516475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f840ad2-d3a2-4a42-a9f0-04a74188b620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.537803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-2f840ad2-d3a2-4a42-a9f0-04a74188b620 None None] MFA Rules not processed for user `0a5c594aafc14c1e86ffa9702f9be964`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:43:51.539008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-2f840ad2-d3a2-4a42-a9f0-04a74188b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:51.546139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1185/2382] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:43:51.559290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a4f5e-ee29-4d00-8f89-7f76ca51e1c0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:51.561145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e05a4f5e-ee29-4d00-8f89-7f76ca51e1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:51.574961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a4f5e-ee29-4d00-8f89-7f76ca51e1c0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:51.578710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828266c-feff-4abe-add6-915fb39866c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:51.579322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828266c-feff-4abe-add6-915fb39866c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.579604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828266c-feff-4abe-add6-915fb39866c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.579904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828266c-feff-4abe-add6-915fb39866c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.587409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828266c-feff-4abe-add6-915fb39866c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:51.588963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828266c-feff-4abe-add6-915fb39866c6 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:51.597800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1198/2383] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:51.702665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1710d-73d1-4dce-9368-0919973924fb None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.703025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1710d-73d1-4dce-9368-0919973924fb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.703351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1710d-73d1-4dce-9368-0919973924fb None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.704825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1186/2384] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:43:51.710514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.710827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.711101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.732227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:43:51.733467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:51.743170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1199/2385] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:43:51.772048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1710d-73d1-4dce-9368-0919973924fb None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:51.784347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1710d-73d1-4dce-9368-0919973924fb None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:51.788613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:51.789219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:51.789485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:51.789748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:51.800830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:51.802177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:51.814229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1187/2386] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:51.912946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:51.914604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:51.994538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d47b7-0e93-4c2d-b679-51344d86b8b0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:51.999961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390c6e7-c727-45bc-be9c-67f69ae29534 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:52.000611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390c6e7-c727-45bc-be9c-67f69ae29534 service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:52.000916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390c6e7-c727-45bc-be9c-67f69ae29534 service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:52.001187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390c6e7-c727-45bc-be9c-67f69ae29534 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:52.003005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9390c6e7-c727-45bc-be9c-67f69ae29534 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:52.013381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:43:52.017856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390c6e7-c727-45bc-be9c-67f69ae29534 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:52.019486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390c6e7-c727-45bc-be9c-67f69ae29534 service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:52.029796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1200/2387] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:53.410190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:53.426515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2ab85-6b9a-4f20-9854-68cc5e505351 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:53.433043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:53.433661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:53.433965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:53.434239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:53.436226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:53.502093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:53.503948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:53.516182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1188/2388] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:43:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:43:54.551666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793967a-14a1-4333-b491-8cf52aa158db None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:54.552192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793967a-14a1-4333-b491-8cf52aa158db None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:54.552192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793967a-14a1-4333-b491-8cf52aa158db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:54.573604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3793967a-14a1-4333-b491-8cf52aa158db None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:43:54.575219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3793967a-14a1-4333-b491-8cf52aa158db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:43:54.583887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1201/2389] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:43:54.596807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:43:54.608640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e5c4d-d724-4bee-8361-d7aa4fb33692 service nova] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:43:54.613619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:43:54.614214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:43:54.614520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:43:54.614803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:43:54.623795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:43:54.625552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:43:54.634426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1189/2390] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:44:04.567384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:44:04.630828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=55 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:44:15.310865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f912b-5556-45f7-90f6-77e50d22a689 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:44:15.311389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f912b-5556-45f7-90f6-77e50d22a689 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:44:15.311389 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f912b-5556-45f7-90f6-77e50d22a689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:44:15.331954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8b0f912b-5556-45f7-90f6-77e50d22a689 None None] MFA Rules not processed for user `9643b0f10e8345889c181efbb1978237`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:44:15.333193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8b0f912b-5556-45f7-90f6-77e50d22a689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:44:15.341064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1202/2391] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:44:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:44:16.890291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:44:16.891967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:44:16.920514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb388a-3467-401d-bd74-e37804ab966a service swift] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:44:16.926980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:44:16.927870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:44:16.928353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:44:16.928682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:44:16.930552 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:44:16.947687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:44:16.949299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:44:16.961054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1190/2392] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:44:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:44:20.668183 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2fc37-0872-4682-8878-44cbcc649c85 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:44:20.668741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2fc37-0872-4682-8878-44cbcc649c85 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:44:20.668741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2fc37-0872-4682-8878-44cbcc649c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:44:20.688074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6bf2fc37-0872-4682-8878-44cbcc649c85 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:44:20.689315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6bf2fc37-0872-4682-8878-44cbcc649c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:44:20.696673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1203/2393] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 22 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:44:20.709927 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:44:20.721324 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b92901-99a4-4a9a-af6d-91bf137fdd4f service nova] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:44:20.726615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:44:20.727254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:44:20.727528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:44:20.727802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:44:20.742339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:44:20.745473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:44:20.758389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1191/2394] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:44:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 05:44:20.795613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2fc37-0872-4682-8878-44cbcc649c85 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:44:20.814428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2fc37-0872-4682-8878-44cbcc649c85 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:44:20.820668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62d8d2-cd35-443c-a91b-5252da29f449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:44:20.821685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62d8d2-cd35-443c-a91b-5252da29f449 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:44:20.822281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62d8d2-cd35-443c-a91b-5252da29f449 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:44:20.822689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62d8d2-cd35-443c-a91b-5252da29f449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:44:20.825479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3e62d8d2-cd35-443c-a91b-5252da29f449 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:44:20.842307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62d8d2-cd35-443c-a91b-5252da29f449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:44:20.844125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62d8d2-cd35-443c-a91b-5252da29f449 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:44:20.866439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1204/2395] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:44:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:44:30.754319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=12 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:44:30.856728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=12 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:10.919167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:10.929283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19201d6-0bd9-4741-aac2-56f5f44891bd service swift] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:10.933802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dca2a-df0b-4d22-8a3f-9ce215a35c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:10.934396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dca2a-df0b-4d22-8a3f-9ce215a35c03 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:10.934668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dca2a-df0b-4d22-8a3f-9ce215a35c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:10.934917 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dca2a-df0b-4d22-8a3f-9ce215a35c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:10.936960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4d9dca2a-df0b-4d22-8a3f-9ce215a35c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:10.946880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dca2a-df0b-4d22-8a3f-9ce215a35c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:10.948282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dca2a-df0b-4d22-8a3f-9ce215a35c03 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:10.956218 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1192/2396] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:45:11.086905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7eeb9-7e24-478b-b854-24c85fa86746 None None] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:11.087527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7eeb9-7e24-478b-b854-24c85fa86746 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:11.087527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7eeb9-7e24-478b-b854-24c85fa86746 None None] PATH_INFO: `/` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:11.088806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1205/2397] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:45:11.095024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b54d9c-e3b6-46c3-8cd8-557d49d788ac None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:11.095443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b54d9c-e3b6-46c3-8cd8-557d49d788ac None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:11.095844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b54d9c-e3b6-46c3-8cd8-557d49d788ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:11.116988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-a8b54d9c-e3b6-46c3-8cd8-557d49d788ac None None] MFA Rules not processed for user `0a5c594aafc14c1e86ffa9702f9be964`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:11.118222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-a8b54d9c-e3b6-46c3-8cd8-557d49d788ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:11.125426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1193/2398] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:45:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:11.138002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7eeb9-7e24-478b-b854-24c85fa86746 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:11.149814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7eeb9-7e24-478b-b854-24c85fa86746 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:11.153491 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d658b-0e27-468e-8f57-9bbb5cb6e390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:11.154090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d658b-0e27-468e-8f57-9bbb5cb6e390 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:11.154357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d658b-0e27-468e-8f57-9bbb5cb6e390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:11.154621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d658b-0e27-468e-8f57-9bbb5cb6e390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:11.162459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d658b-0e27-468e-8f57-9bbb5cb6e390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:11.163918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d658b-0e27-468e-8f57-9bbb5cb6e390 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:11.172390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1206/2399] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:45:11.256813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0f4d7-7c47-4218-914f-8ba506c4a394 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:11.257172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0f4d7-7c47-4218-914f-8ba506c4a394 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:11.257469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0f4d7-7c47-4218-914f-8ba506c4a394 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:11.259436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1194/2400] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:45:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:45:11.264842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97f754-b1ba-4ba8-a718-7c69fbf7c562 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:11.265126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97f754-b1ba-4ba8-a718-7c69fbf7c562 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:11.265395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97f754-b1ba-4ba8-a718-7c69fbf7c562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:11.285858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-ce97f754-b1ba-4ba8-a718-7c69fbf7c562 None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:11.287230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ce97f754-b1ba-4ba8-a718-7c69fbf7c562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:11.298037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1207/2401] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:45:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:45:11.312230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0f4d7-7c47-4218-914f-8ba506c4a394 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:11.330203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0f4d7-7c47-4218-914f-8ba506c4a394 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:11.335718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de502a0-dd93-4ac0-b55c-e54e80b7781a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:11.336310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de502a0-dd93-4ac0-b55c-e54e80b7781a service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:11.336585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de502a0-dd93-4ac0-b55c-e54e80b7781a service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:11.336839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de502a0-dd93-4ac0-b55c-e54e80b7781a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:11.348890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de502a0-dd93-4ac0-b55c-e54e80b7781a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:11.350323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de502a0-dd93-4ac0-b55c-e54e80b7781a service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:11.361441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1195/2402] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:45:21.280727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=9 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:21.356451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=15 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:28.984645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97f754-b1ba-4ba8-a718-7c69fbf7c562 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:28.998694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97f754-b1ba-4ba8-a718-7c69fbf7c562 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:29.004470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:29.005420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:29.005809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:29.006128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:29.007999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:29.095019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:29.097109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:29.119539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1208/2403] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:45:32.234812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646ca07-54e2-41e6-b632-1ac4c2d0370f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:32.235540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646ca07-54e2-41e6-b632-1ac4c2d0370f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:32.235772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646ca07-54e2-41e6-b632-1ac4c2d0370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:32.256220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6646ca07-54e2-41e6-b632-1ac4c2d0370f None None] MFA Rules not processed for user `8e68f520aaff495abfb4b7a5271936e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:32.257407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6646ca07-54e2-41e6-b632-1ac4c2d0370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:32.264669 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1196/2404] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:36.801230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:36.831099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ffd67-c66d-412e-8926-58036d48bfca service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:36.845286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:36.846870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:36.848083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:36.848543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:36.867308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:36.899341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:36.906689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:36.945200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1209/2405] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:45:39.880311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c02d9-c9b8-4160-bd82-9f0728c1037f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:39.880962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c02d9-c9b8-4160-bd82-9f0728c1037f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:39.880962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c02d9-c9b8-4160-bd82-9f0728c1037f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:39.900027 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f91c02d9-c9b8-4160-bd82-9f0728c1037f None None] MFA Rules not processed for user `2864050fd1a64646a8760f0172be8f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:39.901248 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f91c02d9-c9b8-4160-bd82-9f0728c1037f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:39.909202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1197/2406] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:45:46.922744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=34 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:49.895043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=6 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:49.949910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:49.961647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e077a-ab07-4d9c-9255-bfd8ecdeb53b service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:49.966415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:49.967019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:49.967292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:49.967594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:49.969564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:50.038564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:50.040096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:50.050175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1210/2407] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:45:53.708942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c02d9-c9b8-4160-bd82-9f0728c1037f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:53.724541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c02d9-c9b8-4160-bd82-9f0728c1037f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:53.730230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:53.731119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:53.731652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:53.732164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:53.734959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:53.755311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:53.757217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:53.774512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:53.775092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1198/2408] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:45:53.776765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:53.789418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b54a-0041-4bdd-8c10-ebe24474806e service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:53.794064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:53.794700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:53.794976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:53.795246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] PATH_INFO: `/v3/users/9643b0f10e8345889c181efbb1978237` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:53.796405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9643b0f10e8345889c181efbb1978237)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:53.798346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:53.848425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9643b0f10e8345889c181efbb1978237'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:53.848425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9643b0f10e8345889c181efbb1978237'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:53.855414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9643b0f10e8345889c181efbb1978237'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:53.856175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9643b0f10e8345889c181efbb1978237'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:53.865975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-49e39288-16b0-4079-9f4b-16cbca9a9566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9643b0f10e8345889c181efbb1978237'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:53.885141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1211/2409] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:45:53 2022] DELETE /identity/v3/users/9643b0f10e8345889c181efbb1978237 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:54.059053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.072476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d25c1-d665-4392-b414-60ac1c2bb7b7 service placement] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.147734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.148422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.148882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.149318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] PATH_INFO: `/v3/projects/8cadb48afce14e758bfe62771530527a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.150493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cadb48afce14e758bfe62771530527a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.152462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.200021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cadb48afce14e758bfe62771530527a', 'request_id': 'req-63083c67-ebdc-4326-86ee-75131faea951'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:54.205628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cadb48afce14e758bfe62771530527a', 'request_id': 'req-63083c67-ebdc-4326-86ee-75131faea951'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:54.212279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cadb48afce14e758bfe62771530527a', 'request_id': 'req-63083c67-ebdc-4326-86ee-75131faea951'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:54.226601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] The token cache is being invalidate because project 8cadb48afce14e758bfe62771530527a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:45:54.226892 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:54.227496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-63083c67-ebdc-4326-86ee-75131faea951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:54.229281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1199/2410] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:45:54 2022] DELETE /identity/v3/projects/8cadb48afce14e758bfe62771530527a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:54.234123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69bda6-402e-4c3b-87e8-4609cd761da9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.234412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69bda6-402e-4c3b-87e8-4609cd761da9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.234686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69bda6-402e-4c3b-87e8-4609cd761da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.256744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bf69bda6-402e-4c3b-87e8-4609cd761da9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:54.257996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bf69bda6-402e-4c3b-87e8-4609cd761da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:54.321650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1212/2411] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:54.330830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d023-50aa-491f-8d57-6017e752089b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.331120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d023-50aa-491f-8d57-6017e752089b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.331395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d023-50aa-491f-8d57-6017e752089b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.351866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4920d023-50aa-491f-8d57-6017e752089b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:54.353079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4920d023-50aa-491f-8d57-6017e752089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:54.364418 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1200/2412] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:54.368477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69bda6-402e-4c3b-87e8-4609cd761da9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.378901 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69bda6-402e-4c3b-87e8-4609cd761da9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.384174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.384809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.385088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.385506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.386184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:45:54.386487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.388162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.389935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:54.402947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1213/2413] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:45:54.407227 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920d023-50aa-491f-8d57-6017e752089b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.417087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920d023-50aa-491f-8d57-6017e752089b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.422407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.423205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.423668 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.424013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.424870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.427079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.457831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1201/2414] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:45:54 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:45:54.462697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.474392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0d573-63c9-4c66-b73e-2da6ca3dde97 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.535083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.536174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.536636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.536911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.537562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.539521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.564421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1214/2415] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:45:54 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:45:54.569074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.580721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284083e7-c5ef-4e8e-a180-170451401775 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.586169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.586809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.587074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.587332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.588024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.590042 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.601424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1202/2416] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:45:54.605521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.616292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62421aa4-d373-4664-a78f-8b8850d216c6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.621384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f9ff3-49c2-4e54-a2d5-3c4d5cbe38a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.621999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9ff3-49c2-4e54-a2d5-3c4d5cbe38a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.622276 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9ff3-49c2-4e54-a2d5-3c4d5cbe38a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.622542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9ff3-49c2-4e54-a2d5-3c4d5cbe38a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.623189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f9ff3-49c2-4e54-a2d5-3c4d5cbe38a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.624831 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f9ff3-49c2-4e54-a2d5-3c4d5cbe38a6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.639403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1215/2417] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:45:54.640860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.652132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d412a-bc4a-4edb-9039-35b52ec13f10 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.657465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.658256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.658597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.658935 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] PATH_INFO: `/v3/projects/2bdaca62aa20449ab1f146aa46e45a1c/users/d38bcbee836249a78b958a3324fc1db1/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.676275 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdaca62aa20449ab1f146aa46e45a1c, user_id=d38bcbee836249a78b958a3324fc1db1, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.679102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.693984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1203/2418] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:45:54 2022] PUT /identity/v3/projects/2bdaca62aa20449ab1f146aa46e45a1c/users/d38bcbee836249a78b958a3324fc1db1/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:54.698894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3790e-fbe1-4075-9628-c431178a2f2b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.699303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3790e-fbe1-4075-9628-c431178a2f2b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.699604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3790e-fbe1-4075-9628-c431178a2f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.719474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3af3790e-fbe1-4075-9628-c431178a2f2b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:54.720713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3af3790e-fbe1-4075-9628-c431178a2f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:54.784308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1216/2419] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:45:54.795217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:54.798081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:54.813592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:54.860302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e29d2-a8cd-4183-8656-5fe2a30d7a80 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:54.864394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:54.864784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:54.865056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:54.865315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:54.877134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:54.878783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:54.889056 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1204/2420] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:45:55.618230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb7905-763c-4c50-9163-deea49af595e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:55.618546 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb7905-763c-4c50-9163-deea49af595e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:55.618782 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb7905-763c-4c50-9163-deea49af595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:55.649624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6adb7905-763c-4c50-9163-deea49af595e None None] MFA Rules not processed for user `d38bcbee836249a78b958a3324fc1db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:55.650843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6adb7905-763c-4c50-9163-deea49af595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:55.735823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1217/2421] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:55.739546 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:55.749580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03297085-99bb-4612-8572-d2e4c9ad7088 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:55.754385 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:55.755258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:55.755258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:55.755510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:55.756171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:55.758057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:55.794032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1205/2422] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:45:55 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:45:55.798221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb7905-763c-4c50-9163-deea49af595e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:55.804198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:55.810699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb7905-763c-4c50-9163-deea49af595e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:55.868499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:55.869745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:55.870024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:55.870321 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:55.871122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:55.873215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:55.898393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1218/2423] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:45:55 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:45:55.900361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:55.912179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d07a8-7e8c-4ca9-8302-4764d7f50025 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:55.917333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:55.917970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:55.918225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:55.918493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:55.919148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:55.920814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:55.933113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1206/2424] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:45:55.937680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:55.948873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57756b-25aa-47a9-b04a-162431717849 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:55.953906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:55.954595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:55.954872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:55.955142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] PATH_INFO: `/v3/projects/55fd1540f44246faad9e9d4f95dbb3e6/users/ab5ab387f8a54c0baf20378ba477dbfb/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:55.982432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fd1540f44246faad9e9d4f95dbb3e6, user_id=ab5ab387f8a54c0baf20378ba477dbfb, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:55.985168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:56.000376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1219/2425] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:45:55 2022] PUT /identity/v3/projects/55fd1540f44246faad9e9d4f95dbb3e6/users/ab5ab387f8a54c0baf20378ba477dbfb/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:56.005100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:56.016548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee9589-a675-4124-8d4c-e422fa439e86 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:56.077853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c151b3c-d639-4bdc-a44a-7f338edf203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:56.078481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c151b3c-d639-4bdc-a44a-7f338edf203e admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:56.078751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c151b3c-d639-4bdc-a44a-7f338edf203e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:56.079024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c151b3c-d639-4bdc-a44a-7f338edf203e admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:56.079707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c151b3c-d639-4bdc-a44a-7f338edf203e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:56.081360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c151b3c-d639-4bdc-a44a-7f338edf203e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:56.092719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1207/2426] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:45:56.097211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:56.108456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4609f0-9843-4793-93fa-b37654509ee6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:56.114311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:56.114983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:56.115370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:56.115500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] PATH_INFO: `/v3/projects/55fd1540f44246faad9e9d4f95dbb3e6/users/ab5ab387f8a54c0baf20378ba477dbfb/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:56.122960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fd1540f44246faad9e9d4f95dbb3e6, user_id=ab5ab387f8a54c0baf20378ba477dbfb, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:56.125702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:56.137269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1220/2427] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:45:56 2022] PUT /identity/v3/projects/55fd1540f44246faad9e9d4f95dbb3e6/users/ab5ab387f8a54c0baf20378ba477dbfb/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:56.782408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccebdc-b9e5-460d-86e4-91ad06670c81 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:56.782636 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccebdc-b9e5-460d-86e4-91ad06670c81 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:56.782910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccebdc-b9e5-460d-86e4-91ad06670c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:56.811273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f2ccebdc-b9e5-460d-86e4-91ad06670c81 None None] MFA Rules not processed for user `ab5ab387f8a54c0baf20378ba477dbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:56.812531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f2ccebdc-b9e5-460d-86e4-91ad06670c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:56.891904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1208/2428] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:56 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:57.959741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:58.030664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc4a30-98f1-45ec-bae3-c0ec48461151 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:58.035878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:58.036538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:58.036819 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:58.037093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] PATH_INFO: `/v3/users/d38bcbee836249a78b958a3324fc1db1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:58.038273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38bcbee836249a78b958a3324fc1db1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:58.040262 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:58.088628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38bcbee836249a78b958a3324fc1db1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.089646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38bcbee836249a78b958a3324fc1db1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.096572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38bcbee836249a78b958a3324fc1db1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.097274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38bcbee836249a78b958a3324fc1db1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.099456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:58.107603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38bcbee836249a78b958a3324fc1db1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.126454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1221/2429] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:45:57 2022] DELETE /identity/v3/users/d38bcbee836249a78b958a3324fc1db1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:58.312742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccebdc-b9e5-460d-86e4-91ad06670c81 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:58.319766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:58.328835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccebdc-b9e5-460d-86e4-91ad06670c81 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:58.399337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:58.399929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:58.400204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:58.400474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] PATH_INFO: `/v3/projects/2bdaca62aa20449ab1f146aa46e45a1c` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:58.401665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bdaca62aa20449ab1f146aa46e45a1c)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:58.403605 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:58.452676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bdaca62aa20449ab1f146aa46e45a1c', 'request_id': 'req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.458268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bdaca62aa20449ab1f146aa46e45a1c', 'request_id': 'req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.464961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bdaca62aa20449ab1f146aa46e45a1c', 'request_id': 'req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.479760 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] The token cache is being invalidate because project 2bdaca62aa20449ab1f146aa46e45a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:45:58.480050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.480776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.482360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1209/2430] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:45:58 2022] DELETE /identity/v3/projects/2bdaca62aa20449ab1f146aa46e45a1c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:58.486645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:58.488238 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:58.502250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e676-2426-49ec-92ee-5e7acfa74b67 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:58.560599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:58.561227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:58.561506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:58.561759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] PATH_INFO: `/v3/users/ab5ab387f8a54c0baf20378ba477dbfb` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:58.562944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5ab387f8a54c0baf20378ba477dbfb)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:58.564910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:58.614327 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5ab387f8a54c0baf20378ba477dbfb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.614998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5ab387f8a54c0baf20378ba477dbfb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.621793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5ab387f8a54c0baf20378ba477dbfb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.622425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5ab387f8a54c0baf20378ba477dbfb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.631654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-e6cf2989-ccd5-4d22-ba03-7295b7343e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5ab387f8a54c0baf20378ba477dbfb'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.650535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1222/2431] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:45:58 2022] DELETE /identity/v3/users/ab5ab387f8a54c0baf20378ba477dbfb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:58.849370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:58.850838 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:58.866597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb5a58-5817-4f74-ad6c-8fc2eb18543a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:58.928611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:58.929254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:58.929536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:58.929809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] PATH_INFO: `/v3/projects/55fd1540f44246faad9e9d4f95dbb3e6` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:58.931080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55fd1540f44246faad9e9d4f95dbb3e6)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:58.933155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:58.970992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:58.984531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55fd1540f44246faad9e9d4f95dbb3e6', 'request_id': 'req-d03409ed-59b6-47cd-ac65-30b400ca7586'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.990924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55fd1540f44246faad9e9d4f95dbb3e6', 'request_id': 'req-d03409ed-59b6-47cd-ac65-30b400ca7586'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:58.997571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55fd1540f44246faad9e9d4f95dbb3e6', 'request_id': 'req-d03409ed-59b6-47cd-ac65-30b400ca7586'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:59.013473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] The token cache is being invalidate because project 55fd1540f44246faad9e9d4f95dbb3e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:45:59.013760 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:59.014444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d03409ed-59b6-47cd-ac65-30b400ca7586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:45:59.015997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1210/2432] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:45:58 2022] DELETE /identity/v3/projects/55fd1540f44246faad9e9d4f95dbb3e6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:59.021248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd9713-8a49-4155-a44a-57ee36e3f685 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.021748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd9713-8a49-4155-a44a-57ee36e3f685 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.021862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd9713-8a49-4155-a44a-57ee36e3f685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.042834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6fcd9713-8a49-4155-a44a-57ee36e3f685 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:59.044345 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6fcd9713-8a49-4155-a44a-57ee36e3f685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:59.103601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:59.118645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1223/2433] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:59.131403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9901-7442-48f3-aec0-cf5ca8b536f6 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.131717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9901-7442-48f3-aec0-cf5ca8b536f6 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.131988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9901-7442-48f3-aec0-cf5ca8b536f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.152601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-3f2e9901-7442-48f3-aec0-cf5ca8b536f6 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:59.153799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3f2e9901-7442-48f3-aec0-cf5ca8b536f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:59.163180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1211/2434] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:59.168283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd9713-8a49-4155-a44a-57ee36e3f685 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.184643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd9713-8a49-4155-a44a-57ee36e3f685 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.193555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.194227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.194546 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.194865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.195523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:45:59.195851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.197516 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.209927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1224/2435] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:45:59.214973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9901-7442-48f3-aec0-cf5ca8b536f6 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.224847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9901-7442-48f3-aec0-cf5ca8b536f6 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.229842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.230494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.230753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.231046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.231715 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.233612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.263032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1212/2436] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:45:59 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:45:59.267359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.277779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64876b-a44f-47c8-b03c-74cc97a8ba62 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.332323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.332918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.333428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.333767 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.334505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.336473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.362142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1225/2437] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:45:59 2022] POST /identity/v3/users => generated 304 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:45:59.366495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.378973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2143c-203e-4a14-9b57-b4335c64ad52 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.384923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.385762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.386192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.386616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.387277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.389859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.403053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1213/2438] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:45:59.408409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.421273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4844212-58fd-429b-9558-8334de23e760 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.428973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb892d7f-8c6e-4e50-982d-13fc7a679785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.429592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb892d7f-8c6e-4e50-982d-13fc7a679785 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.429876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb892d7f-8c6e-4e50-982d-13fc7a679785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.430143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb892d7f-8c6e-4e50-982d-13fc7a679785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.430798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb892d7f-8c6e-4e50-982d-13fc7a679785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.432470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb892d7f-8c6e-4e50-982d-13fc7a679785 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.443993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1226/2439] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:45:59.448878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.462206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60dce7-deb3-45ff-9fc0-643f25c3dffd admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.467481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.468184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.468461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.468738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] PATH_INFO: `/v3/projects/90055eaa40ea436e894b4d08f93cb665/users/a8394a7289f14fbbb9d53ca61d32bd45/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.484429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90055eaa40ea436e894b4d08f93cb665, user_id=a8394a7289f14fbbb9d53ca61d32bd45, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.487091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.502023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1214/2440] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:45:59 2022] PUT /identity/v3/projects/90055eaa40ea436e894b4d08f93cb665/users/a8394a7289f14fbbb9d53ca61d32bd45/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:59.512371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8c554-24b6-4dc1-b32e-029219322d74 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.512658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8c554-24b6-4dc1-b32e-029219322d74 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.512937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8c554-24b6-4dc1-b32e-029219322d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.541328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-05c8c554-24b6-4dc1-b32e-029219322d74 None None] MFA Rules not processed for user `a8394a7289f14fbbb9d53ca61d32bd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:45:59.542571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-05c8c554-24b6-4dc1-b32e-029219322d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:45:59.608181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1227/2441] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:45:59 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:45:59.612733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.677283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a830-611f-494b-91aa-6702c3ad8239 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.682840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.683442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.683912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.684184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.684805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.686718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.716740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1215/2442] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:45:59 2022] POST /identity/v3/projects => generated 407 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:45:59.721052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8c554-24b6-4dc1-b32e-029219322d74 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.732716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8c554-24b6-4dc1-b32e-029219322d74 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.788791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:45:59.804063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.804754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.805037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.805301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.805925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.807970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.833244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1228/2443] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:45:59 2022] POST /identity/v3/users => generated 310 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:45:59.837626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.850863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5ba4-62f8-45f1-b02d-bd6518ac5a46 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.855781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.856443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.856731 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.857004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.857639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.859257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.869844 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1216/2444] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:45:59.874249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.886710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112f8f4-994c-48b2-96a5-3e8676d14b1e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.891783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:45:59.892444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:45:59.892717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:45:59.892986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] PATH_INFO: `/v3/projects/e4ce0c910b96479989073fe011d31121/users/bedcf831c77049c8ad135085762ab27c/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:45:59.909142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ce0c910b96479989073fe011d31121, user_id=bedcf831c77049c8ad135085762ab27c, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:45:59.912153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:45:59.925917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1229/2445] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:45:59 2022] PUT /identity/v3/projects/e4ce0c910b96479989073fe011d31121/users/bedcf831c77049c8ad135085762ab27c/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:45:59.930165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:45:59.940319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b69b0-6f47-41fb-af08-96b2c2433ef3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:45:59.975687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:00.003167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ef970-5234-4b13-8286-d054bc5a6f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:00.003817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef970-5234-4b13-8286-d054bc5a6f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:00.004098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef970-5234-4b13-8286-d054bc5a6f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:00.004367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef970-5234-4b13-8286-d054bc5a6f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:00.005116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ef970-5234-4b13-8286-d054bc5a6f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:00.007096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ef970-5234-4b13-8286-d054bc5a6f15 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:00.017481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1217/2446] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:00.022091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:00.032681 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a579299-3b07-4def-8725-d5df7e138291 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:00.037773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:00.038426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:00.038728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:00.039002 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] PATH_INFO: `/v3/projects/e4ce0c910b96479989073fe011d31121/users/bedcf831c77049c8ad135085762ab27c/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:00.040985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ce0c910b96479989073fe011d31121, user_id=bedcf831c77049c8ad135085762ab27c, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:00.043665 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:00.054379 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1230/2447] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:00 2022] PUT /identity/v3/projects/e4ce0c910b96479989073fe011d31121/users/bedcf831c77049c8ad135085762ab27c/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:00.064948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682dd89c-9afb-4018-92c4-8c03308c61b0 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:00.065232 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682dd89c-9afb-4018-92c4-8c03308c61b0 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:00.065507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682dd89c-9afb-4018-92c4-8c03308c61b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:00.093675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-682dd89c-9afb-4018-92c4-8c03308c61b0 None None] MFA Rules not processed for user `bedcf831c77049c8ad135085762ab27c`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:00.094883 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-682dd89c-9afb-4018-92c4-8c03308c61b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:00.161690 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1218/2448] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:00.172723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:00.174792 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:00.235199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d8ab0-2853-413e-a80a-2de8a93361fd admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:00.240731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:00.241338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:00.241607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:00.241869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:00.251064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:00.252537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:00.262121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1231/2449] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:00.423022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682dd89c-9afb-4018-92c4-8c03308c61b0 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:00.433293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682dd89c-9afb-4018-92c4-8c03308c61b0 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:00.437197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29c918-351c-48e7-a8fa-1ac1b8e818de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:00.437811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29c918-351c-48e7-a8fa-1ac1b8e818de service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:00.438088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29c918-351c-48e7-a8fa-1ac1b8e818de service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:00.438352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29c918-351c-48e7-a8fa-1ac1b8e818de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:00.499537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29c918-351c-48e7-a8fa-1ac1b8e818de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:00.500938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29c918-351c-48e7-a8fa-1ac1b8e818de service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:00.510292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1219/2450] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:00 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:10.045056 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:10.109131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fa108-971a-4d92-b747-049780dd09d8 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:10.113919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:10.114549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:10.114826 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:10.115097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] PATH_INFO: `/v3/users/a8394a7289f14fbbb9d53ca61d32bd45` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:10.116265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8394a7289f14fbbb9d53ca61d32bd45)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:10.118311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:10.171408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:10.173335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8394a7289f14fbbb9d53ca61d32bd45'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.174020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8394a7289f14fbbb9d53ca61d32bd45'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.182405 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8394a7289f14fbbb9d53ca61d32bd45'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.183271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8394a7289f14fbbb9d53ca61d32bd45'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.191526 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8394a7289f14fbbb9d53ca61d32bd45'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.209009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1232/2451] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:10 2022] DELETE /identity/v3/users/a8394a7289f14fbbb9d53ca61d32bd45 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:10.213783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:10.214644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:10.214750 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:10.234298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:10.235597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:10.286705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:10.304499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1220/2452] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:10.316780 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:10.319243 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:10.385948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be95c33-1117-4cd8-acbc-24be8d8fab5c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:10.391940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:10.392944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:10.393398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:10.393899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:10.409290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:10.411328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:10.423116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1233/2453] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:10.712752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:10.714557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:10.726512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1b690-cb43-4164-a1eb-171b79b1691f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:10.731809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:10.732644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:10.733028 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:10.733503 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] PATH_INFO: `/v3/projects/90055eaa40ea436e894b4d08f93cb665` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:10.734659 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90055eaa40ea436e894b4d08f93cb665)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:10.736974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:10.787358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90055eaa40ea436e894b4d08f93cb665', 'request_id': 'req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.794082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90055eaa40ea436e894b4d08f93cb665', 'request_id': 'req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.801151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90055eaa40ea436e894b4d08f93cb665', 'request_id': 'req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.815424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] The token cache is being invalidate because project 90055eaa40ea436e894b4d08f93cb665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:10.815995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.816637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.821460 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1221/2454] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:10 2022] DELETE /identity/v3/projects/90055eaa40ea436e894b4d08f93cb665 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:10.822690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:10.832868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005f0e0-775e-4b1b-a0a5-50106dfcd191 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:10.888931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:10.889569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:10.890083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:10.890361 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] PATH_INFO: `/v3/users/bedcf831c77049c8ad135085762ab27c` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:10.891479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bedcf831c77049c8ad135085762ab27c)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:10.893646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:10.951655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bedcf831c77049c8ad135085762ab27c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.952475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bedcf831c77049c8ad135085762ab27c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.959712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bedcf831c77049c8ad135085762ab27c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.960377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bedcf831c77049c8ad135085762ab27c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.969426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c377b6cb-9b70-4e03-8c99-b2a396c958b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bedcf831c77049c8ad135085762ab27c'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:10.973987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:10.988944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1234/2455] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:10 2022] DELETE /identity/v3/users/bedcf831c77049c8ad135085762ab27c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:46:11.268675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:11.270302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:11.286104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5cbd6-30c0-45ff-88f1-7696c4068c66 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:11.355055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:11.355733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.356010 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.356279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] PATH_INFO: `/v3/projects/e4ce0c910b96479989073fe011d31121` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.357434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4ce0c910b96479989073fe011d31121)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:11.359388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:11.411657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4ce0c910b96479989073fe011d31121', 'request_id': 'req-a84b7bd9-0de8-463f-8b5e-db46d634954f'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:11.421504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4ce0c910b96479989073fe011d31121', 'request_id': 'req-a84b7bd9-0de8-463f-8b5e-db46d634954f'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:11.431473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4ce0c910b96479989073fe011d31121', 'request_id': 'req-a84b7bd9-0de8-463f-8b5e-db46d634954f'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:11.444587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] The token cache is being invalidate because project e4ce0c910b96479989073fe011d31121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:11.444961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:11.445687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a84b7bd9-0de8-463f-8b5e-db46d634954f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:11.447182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1222/2456] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:11 2022] DELETE /identity/v3/projects/e4ce0c910b96479989073fe011d31121 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:11.452354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641565a-281f-48ec-a93c-29b85d9a59ac None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.452864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641565a-281f-48ec-a93c-29b85d9a59ac None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.452978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641565a-281f-48ec-a93c-29b85d9a59ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.473102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8641565a-281f-48ec-a93c-29b85d9a59ac None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:11.474309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8641565a-281f-48ec-a93c-29b85d9a59ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:11.535190 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1235/2457] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:11.543083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fab8-f438-4db7-a651-3008290fd8f1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.543366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fab8-f438-4db7-a651-3008290fd8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.543672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fab8-f438-4db7-a651-3008290fd8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.565334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6199fab8-f438-4db7-a651-3008290fd8f1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:11.566518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6199fab8-f438-4db7-a651-3008290fd8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:11.574907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1223/2458] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:11.579463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641565a-281f-48ec-a93c-29b85d9a59ac None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:11.590872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641565a-281f-48ec-a93c-29b85d9a59ac None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:11.596490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:11.597094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.597378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.597645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.598290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:46:11.598601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:11.600306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:11.613530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1236/2459] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:46:11.618389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fab8-f438-4db7-a651-3008290fd8f1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:11.622804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:11.628718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fab8-f438-4db7-a651-3008290fd8f1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:11.635294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:11.636140 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.636610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.637063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.637719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:11.639610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:11.669422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1224/2460] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:11 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:46:11.673687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:11.684406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d918b1-cf38-4475-bf7b-080ab71517a2 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:11.745175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:11.745795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.746073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.746343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.746967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:11.748938 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:11.772523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1237/2461] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:11 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:11.776790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:11.787293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4d89c-fb88-4b72-9e08-25485d3516a6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:11.792303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:11.792893 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.793163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.793427 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.794047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:11.795663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:11.806358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1225/2462] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:11.810496 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:11.821460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24057c56-e7ba-4db4-9b91-de248986de32 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:11.827313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff86b3-b26c-429a-8e95-2a395fc1213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:11.827946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff86b3-b26c-429a-8e95-2a395fc1213e admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.828223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff86b3-b26c-429a-8e95-2a395fc1213e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.828493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff86b3-b26c-429a-8e95-2a395fc1213e admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.829140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff86b3-b26c-429a-8e95-2a395fc1213e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:11.830754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff86b3-b26c-429a-8e95-2a395fc1213e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:11.843422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1238/2463] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:11.847821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:11.858328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0598f89-a9ca-4b8d-94de-19014face0ba admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:11.863718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:11.864372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.864644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.864908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] PATH_INFO: `/v3/projects/963da4041ed045809a31f32ee8d9a607/users/9ea0370eb53b47d8b5216a2c15ed42dc/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.881353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963da4041ed045809a31f32ee8d9a607, user_id=9ea0370eb53b47d8b5216a2c15ed42dc, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:11.884198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:11.899311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1226/2464] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:11 2022] PUT /identity/v3/projects/963da4041ed045809a31f32ee8d9a607/users/9ea0370eb53b47d8b5216a2c15ed42dc/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:11.910647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9066e-cad0-4c5c-8825-cf3e972f4e58 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:11.910647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9066e-cad0-4c5c-8825-cf3e972f4e58 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:11.910647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9066e-cad0-4c5c-8825-cf3e972f4e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:11.940603 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2db9066e-cad0-4c5c-8825-cf3e972f4e58 None None] MFA Rules not processed for user `9ea0370eb53b47d8b5216a2c15ed42dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:11.941740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2db9066e-cad0-4c5c-8825-cf3e972f4e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:11.988126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:12.022763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1239/2465] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:12.028113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.104471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162d600-8d86-410e-9202-058173c39af4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.110038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.110641 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.110921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.111191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.111835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.113753 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.142906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1227/2466] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:12 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:12.147590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9066e-cad0-4c5c-8825-cf3e972f4e58 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.159876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9066e-cad0-4c5c-8825-cf3e972f4e58 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.217898 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.218428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.218700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.218972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.219619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.221568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.242981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1240/2467] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:12 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:12.247285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.257898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c428-095f-4223-a824-a6bedaa7c8c7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.263806 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.264403 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.264684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.264949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.265594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.267195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.278071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1228/2468] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:12.282187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.293415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705649c5-82d7-4c7b-b3db-ca5fca2c1e48 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.298609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b6b8d-e9f8-4f13-9e27-1d5aaf966653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.299213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b8d-e9f8-4f13-9e27-1d5aaf966653 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.299492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b8d-e9f8-4f13-9e27-1d5aaf966653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.299801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b8d-e9f8-4f13-9e27-1d5aaf966653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.300484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b6b8d-e9f8-4f13-9e27-1d5aaf966653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.302122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b6b8d-e9f8-4f13-9e27-1d5aaf966653 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.312450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1241/2469] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:12.316696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.328256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7de799-31a8-4395-b8af-5bc1eaecb24f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.332952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.333607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.333885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.334154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] PATH_INFO: `/v3/projects/f97d09f20bc54d76af5570fab43433c8/users/52e7336240984d44aebb0db98ee3ac39/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.357282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97d09f20bc54d76af5570fab43433c8, user_id=52e7336240984d44aebb0db98ee3ac39, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.359951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.378859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1229/2470] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:12 2022] PUT /identity/v3/projects/f97d09f20bc54d76af5570fab43433c8/users/52e7336240984d44aebb0db98ee3ac39/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:12.390570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced48e-6de1-4cbf-b1e7-e3e07518a93c None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.390818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced48e-6de1-4cbf-b1e7-e3e07518a93c None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.391109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced48e-6de1-4cbf-b1e7-e3e07518a93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.421111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-ccced48e-6de1-4cbf-b1e7-e3e07518a93c None None] MFA Rules not processed for user `52e7336240984d44aebb0db98ee3ac39`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:12.422317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ccced48e-6de1-4cbf-b1e7-e3e07518a93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:12.496865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1242/2471] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:12.501467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.562959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:12.568095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2efaf8-4608-4940-a1d8-8674329617d2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.574033 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.574648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.574933 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.575464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.576159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.578188 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.612888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1230/2472] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:12 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:12.617409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced48e-6de1-4cbf-b1e7-e3e07518a93c None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.633560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced48e-6de1-4cbf-b1e7-e3e07518a93c None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.681436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:12.696035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.696649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.696932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.697205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.697837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.699799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.725866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1243/2473] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:12 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:12.728008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.742932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842b930-a4c1-4e31-a247-c0da01566f4f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.748420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.748964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.749237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.749501 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.750144 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.751783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.762593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1231/2474] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:12.766872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.777671 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf3aaf-36c2-4eeb-a7d8-75961e798598 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.782551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.783200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.783480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.783784 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] PATH_INFO: `/v3/projects/a9bdece31ffd401db6aecd307f8b5188/users/9ee5f105fe884156b44a07587aec1b1f/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.799899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bdece31ffd401db6aecd307f8b5188, user_id=9ee5f105fe884156b44a07587aec1b1f, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.802577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.816347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1244/2475] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:12 2022] PUT /identity/v3/projects/a9bdece31ffd401db6aecd307f8b5188/users/9ee5f105fe884156b44a07587aec1b1f/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:12.820413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.830860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e92c7-4804-411f-9e21-2b4f1841105b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.889629 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5b908-4bf3-4e66-b038-fb9d41af1ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.890240 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5b908-4bf3-4e66-b038-fb9d41af1ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.890511 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5b908-4bf3-4e66-b038-fb9d41af1ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.890773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5b908-4bf3-4e66-b038-fb9d41af1ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.891407 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5b908-4bf3-4e66-b038-fb9d41af1ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.893049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5b908-4bf3-4e66-b038-fb9d41af1ae3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.903465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1232/2476] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:12.908054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:12.918660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79f66e-1736-4d67-b3fd-80f0163d1a40 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:12.923863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:12.924515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.924795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.925067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] PATH_INFO: `/v3/projects/a9bdece31ffd401db6aecd307f8b5188/users/9ee5f105fe884156b44a07587aec1b1f/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.927114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bdece31ffd401db6aecd307f8b5188, user_id=9ee5f105fe884156b44a07587aec1b1f, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:12.930104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:12.940576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1245/2477] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:12 2022] PUT /identity/v3/projects/a9bdece31ffd401db6aecd307f8b5188/users/9ee5f105fe884156b44a07587aec1b1f/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:12.950765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:12.951075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:12.951342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:12.979718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] MFA Rules not processed for user `9ee5f105fe884156b44a07587aec1b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:12.980908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:13.055040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1233/2478] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:46:13.066923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:13.068824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:13.132453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1d76f-558f-47a3-8aaf-6afb8857d3a3 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:13.136163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:13.136768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:13.137044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:13.137313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:13.146421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:13.147910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:13.157432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1246/2479] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:13.205539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:13.207002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:13.219780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac08a-d89b-4bdf-815b-b1458b6d506c None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:13.223530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b325c3e-4ec3-47c9-816b-4358f514aa94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:13.224185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325c3e-4ec3-47c9-816b-4358f514aa94 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:13.224454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325c3e-4ec3-47c9-816b-4358f514aa94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:13.224718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325c3e-4ec3-47c9-816b-4358f514aa94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:13.272206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:13.282450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b325c3e-4ec3-47c9-816b-4358f514aa94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:13.284132 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b325c3e-4ec3-47c9-816b-4358f514aa94 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:13.292973 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1234/2480] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:16.282590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:16.404356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9674159-891d-40de-902c-e9368bfb2bc9 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:16.409732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:16.410354 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:16.410626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:16.410902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] PATH_INFO: `/v3/users/9ea0370eb53b47d8b5216a2c15ed42dc` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:16.412050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea0370eb53b47d8b5216a2c15ed42dc)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:16.413997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:16.416511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:16.465059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea0370eb53b47d8b5216a2c15ed42dc'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:16.465668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea0370eb53b47d8b5216a2c15ed42dc'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:16.472294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea0370eb53b47d8b5216a2c15ed42dc'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:16.472911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea0370eb53b47d8b5216a2c15ed42dc'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:16.481855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea0370eb53b47d8b5216a2c15ed42dc'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:16.509123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1247/2481] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:16 2022] DELETE /identity/v3/users/9ea0370eb53b47d8b5216a2c15ed42dc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:16.513728 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:16.514235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:16.514235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:16.565776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:16.566993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:16.634544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1235/2482] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:16.646609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:16.648787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:16.724959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e9c3-9ec8-446c-b033-aab1d0efe824 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:16.729473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:16.730083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:16.730367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:16.730634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:16.740107 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:16.741571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:16.751421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1248/2483] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:17.050262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:17.051722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:17.065191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bbafe-6a20-4a93-9181-22c40c739436 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:17.070512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:17.071308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:17.071615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:17.071881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] PATH_INFO: `/v3/projects/963da4041ed045809a31f32ee8d9a607` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:17.079925 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963da4041ed045809a31f32ee8d9a607)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:17.081891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:17.136125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963da4041ed045809a31f32ee8d9a607', 'request_id': 'req-f6cc26a6-432c-4014-972d-625a514dd3e6'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.142353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963da4041ed045809a31f32ee8d9a607', 'request_id': 'req-f6cc26a6-432c-4014-972d-625a514dd3e6'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.148721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963da4041ed045809a31f32ee8d9a607', 'request_id': 'req-f6cc26a6-432c-4014-972d-625a514dd3e6'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.164065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] The token cache is being invalidate because project 963da4041ed045809a31f32ee8d9a607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:17.164348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.165204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.166730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1236/2484] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:17 2022] DELETE /identity/v3/projects/963da4041ed045809a31f32ee8d9a607 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:17.170970 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:17.181641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e6a39-2035-40e1-8fdf-8336d198bc58 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:17.241967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:17.242517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:17.243149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:17.243149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] PATH_INFO: `/v3/users/52e7336240984d44aebb0db98ee3ac39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:17.244281 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e7336240984d44aebb0db98ee3ac39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:17.246215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:17.256989 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:17.299849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e7336240984d44aebb0db98ee3ac39'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.300524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e7336240984d44aebb0db98ee3ac39'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.309176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e7336240984d44aebb0db98ee3ac39'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.309865 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e7336240984d44aebb0db98ee3ac39'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.318215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e7336240984d44aebb0db98ee3ac39'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.335996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1249/2485] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:17 2022] DELETE /identity/v3/users/52e7336240984d44aebb0db98ee3ac39 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:17.626874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:17.628926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:17.643413 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc26a6-432c-4014-972d-625a514dd3e6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:17.690863 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:17.706857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:17.707499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:17.707866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:17.708128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] PATH_INFO: `/v3/projects/f97d09f20bc54d76af5570fab43433c8` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:17.709259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97d09f20bc54d76af5570fab43433c8)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:17.711192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:17.759001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97d09f20bc54d76af5570fab43433c8', 'request_id': 'req-36ace7ee-73f4-4e70-8338-0a999811c162'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.765474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97d09f20bc54d76af5570fab43433c8', 'request_id': 'req-36ace7ee-73f4-4e70-8338-0a999811c162'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.772040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97d09f20bc54d76af5570fab43433c8', 'request_id': 'req-36ace7ee-73f4-4e70-8338-0a999811c162'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.786849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] The token cache is being invalidate because project f97d09f20bc54d76af5570fab43433c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:17.787136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.787797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.789318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1237/2486] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:17 2022] DELETE /identity/v3/projects/f97d09f20bc54d76af5570fab43433c8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:17.794027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:17.795429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:17.808585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bf47c-c290-4b88-99c2-6548b8c50a37 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:17.864858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:17.865485 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:17.865759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:17.866027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] PATH_INFO: `/v3/users/9ee5f105fe884156b44a07587aec1b1f` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:17.867406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee5f105fe884156b44a07587aec1b1f)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:17.869429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:17.923900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee5f105fe884156b44a07587aec1b1f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.924654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee5f105fe884156b44a07587aec1b1f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.932075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee5f105fe884156b44a07587aec1b1f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.932772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee5f105fe884156b44a07587aec1b1f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.941118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f8fa7a9d-3b4e-4df9-bab2-3e2a568fc6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee5f105fe884156b44a07587aec1b1f'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:17.958380 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1250/2487] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:17 2022] DELETE /identity/v3/users/9ee5f105fe884156b44a07587aec1b1f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:18.252005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:18.253539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:18.266897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ace7ee-73f4-4e70-8338-0a999811c162 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:18.331045 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:18.331595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.331903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.332167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] PATH_INFO: `/v3/projects/a9bdece31ffd401db6aecd307f8b5188` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.333291 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9bdece31ffd401db6aecd307f8b5188)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:18.335692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:18.386224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9bdece31ffd401db6aecd307f8b5188', 'request_id': 'req-d9adf4fb-a829-4d01-bd77-68187302c5e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:18.393932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9bdece31ffd401db6aecd307f8b5188', 'request_id': 'req-d9adf4fb-a829-4d01-bd77-68187302c5e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:18.403003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9bdece31ffd401db6aecd307f8b5188', 'request_id': 'req-d9adf4fb-a829-4d01-bd77-68187302c5e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:18.420026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] The token cache is being invalidate because project a9bdece31ffd401db6aecd307f8b5188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:18.420309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:18.420921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9adf4fb-a829-4d01-bd77-68187302c5e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:18.422452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1238/2488] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:18 2022] DELETE /identity/v3/projects/a9bdece31ffd401db6aecd307f8b5188 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:18.429027 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e13fb0-ffb9-4706-9bd5-edd539b9ef42 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.429545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e13fb0-ffb9-4706-9bd5-edd539b9ef42 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.429661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e13fb0-ffb9-4706-9bd5-edd539b9ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.440162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:18.488414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-31e13fb0-ffb9-4706-9bd5-edd539b9ef42 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:18.489639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-31e13fb0-ffb9-4706-9bd5-edd539b9ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:18.553993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1251/2489] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:18.566948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5b7e6-ba66-43cb-8d25-b693809f36d0 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.567303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5b7e6-ba66-43cb-8d25-b693809f36d0 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.567651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5b7e6-ba66-43cb-8d25-b693809f36d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.583529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:18.591034 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-30e5b7e6-ba66-43cb-8d25-b693809f36d0 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:18.593283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-30e5b7e6-ba66-43cb-8d25-b693809f36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:18.603988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1239/2490] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:18.608652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e13fb0-ffb9-4706-9bd5-edd539b9ef42 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:18.619552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e13fb0-ffb9-4706-9bd5-edd539b9ef42 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:18.624869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:18.625472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.625754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.626024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.626697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:46:18.626978 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:18.628626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:18.642269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1252/2491] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:46:18.647652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5b7e6-ba66-43cb-8d25-b693809f36d0 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:18.662079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5b7e6-ba66-43cb-8d25-b693809f36d0 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:18.668924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:18.669541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.669822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.670093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.670729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:18.672705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:18.703547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1240/2492] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:18 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:18.708039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:18.719592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d13ef-dc96-4265-ba79-5525aeafd59c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:18.778453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:18.779062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.779342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.779651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.780278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:18.782231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:18.808169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1253/2493] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:18 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:18.812947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:18.829814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a6152-89a3-4c83-ba0d-52174af7278c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:18.837801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:18.838335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.838614 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.839224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.839888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:18.841498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:18.851645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1241/2494] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:18.856009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:18.866303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a4bee-204b-413d-bf46-bf583da9fd81 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:18.872022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db03ed2-6797-48e8-aef5-2a1ddf0c8bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:18.872627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03ed2-6797-48e8-aef5-2a1ddf0c8bed admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.872907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03ed2-6797-48e8-aef5-2a1ddf0c8bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.873180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03ed2-6797-48e8-aef5-2a1ddf0c8bed admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.873835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db03ed2-6797-48e8-aef5-2a1ddf0c8bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:18.875476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db03ed2-6797-48e8-aef5-2a1ddf0c8bed admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:18.886565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1254/2495] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:18.892877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:18.904180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ec04f-fedb-4394-94da-0e5585ac1db4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:18.909304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:18.909998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.910274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.910542 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] PATH_INFO: `/v3/projects/4fd13262be8b4d1e9c660452ec33a983/users/798ec8047b564127b4249840e00c939b/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.926191 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd13262be8b4d1e9c660452ec33a983, user_id=798ec8047b564127b4249840e00c939b, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:18.928907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:18.943398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1242/2496] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:18 2022] PUT /identity/v3/projects/4fd13262be8b4d1e9c660452ec33a983/users/798ec8047b564127b4249840e00c939b/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:18.949008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ff1ea-e95c-4439-8265-7c37a3d10ff0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:18.949291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ff1ea-e95c-4439-8265-7c37a3d10ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:18.949559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ff1ea-e95c-4439-8265-7c37a3d10ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:18.973014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5b0ff1ea-e95c-4439-8265-7c37a3d10ff0 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:18.974907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5b0ff1ea-e95c-4439-8265-7c37a3d10ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:19.028640 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:19.046480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1255/2497] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:19.056878 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:19.058908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:19.124153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81773857-c5b2-4dcd-a33b-cf11e700b351 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:19.131185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:19.132128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:19.132410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:19.132649 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:19.142585 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:19.144061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:19.153454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1243/2498] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:19.823216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b616d-1051-4d8e-8675-ebd9640a31b0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:19.823801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b616d-1051-4d8e-8675-ebd9640a31b0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:19.823801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b616d-1051-4d8e-8675-ebd9640a31b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:19.855684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-221b616d-1051-4d8e-8675-ebd9640a31b0 None None] MFA Rules not processed for user `798ec8047b564127b4249840e00c939b`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:19.856920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-221b616d-1051-4d8e-8675-ebd9640a31b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:19.940460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1256/2499] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:19.945118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:19.955107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa10a17-ac95-49d5-8a07-b9020438ea5f service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:19.959903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:19.960502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:19.960776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:19.961026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:19.961912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:19.964258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:19.994071 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1244/2500] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:19 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:19.998186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b616d-1051-4d8e-8675-ebd9640a31b0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:20.009515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b616d-1051-4d8e-8675-ebd9640a31b0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:20.065435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:20.066040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:20.066346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:20.066613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:20.067235 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:20.069248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:20.093120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1257/2501] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:19 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:20.100210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:20.110416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b17c0-4508-4f10-8ebc-8aa96ef691a8 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:20.115068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:20.115705 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:20.115981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:20.116247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:20.116886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:20.118733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:20.130327 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1245/2502] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:20.134623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:20.145078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368b1ef-e91f-49d9-b5ef-1af58dcb0516 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:20.150300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:20.150951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:20.151227 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:20.151493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] PATH_INFO: `/v3/projects/d0cf90a269cb40d4bf975a0ce855f4f1/users/a516aa46b3db48b89805f90f9ddd3f96/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:20.168642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cf90a269cb40d4bf975a0ce855f4f1, user_id=a516aa46b3db48b89805f90f9ddd3f96, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:20.172161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:20.190756 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1258/2503] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:20 2022] PUT /identity/v3/projects/d0cf90a269cb40d4bf975a0ce855f4f1/users/a516aa46b3db48b89805f90f9ddd3f96/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:20.194778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:20.206809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14886b-bbc3-4d84-972d-2a671ee092f1 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:20.273662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3a83-935d-4ce4-a301-11d05a24fd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:20.274417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3a83-935d-4ce4-a301-11d05a24fd9b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:20.274696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3a83-935d-4ce4-a301-11d05a24fd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:20.274963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3a83-935d-4ce4-a301-11d05a24fd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:20.275624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a3a83-935d-4ce4-a301-11d05a24fd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:20.277242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a3a83-935d-4ce4-a301-11d05a24fd9b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:20.287686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1246/2504] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:20.292915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:20.307606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7d27-c3dd-4b47-95ae-6b3c445fd95a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:20.315829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:20.316700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:20.316971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:20.317210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] PATH_INFO: `/v3/projects/d0cf90a269cb40d4bf975a0ce855f4f1/users/a516aa46b3db48b89805f90f9ddd3f96/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:20.320450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cf90a269cb40d4bf975a0ce855f4f1, user_id=a516aa46b3db48b89805f90f9ddd3f96, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:20.324914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:20.340010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1259/2505] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:20 2022] PUT /identity/v3/projects/d0cf90a269cb40d4bf975a0ce855f4f1/users/a516aa46b3db48b89805f90f9ddd3f96/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:20.984481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:20.985049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:20.985049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:21.005269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:21.016992 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] MFA Rules not processed for user `a516aa46b3db48b89805f90f9ddd3f96`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:21.018307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:21.094480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1247/2506] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:21.106653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:21.108510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:21.143435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:21.169366 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b076a-d49f-47ce-8c2d-3b0a73451d99 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:21.173088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:21.173698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:21.173974 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:21.174269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:21.230540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:21.232089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:21.241621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1260/2507] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:22.626250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:22.627952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:22.641730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5d2e-fadd-45ea-8bd7-b65ed9690eda None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:22.646042 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:22.646646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:22.646920 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:22.647190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:22.657274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:22.658707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:22.668509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1248/2508] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:27.358530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:27.360073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:27.455722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684108d7-927e-411c-9ec0-5dd7e0c2d280 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:27.461063 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:27.461680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:27.461956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:27.462296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:27.464848 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:27.477154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:27.478751 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:27.489406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1261/2509] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:32.664301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=36 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:35.998792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:36.067215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9ea33-bd2d-48ad-944c-93528ba82098 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:36.072068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:36.072744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:36.073022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:36.073300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] PATH_INFO: `/v3/users/798ec8047b564127b4249840e00c939b` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:36.074510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] RBAC: Authorizing `identity:delete_user(user_id=798ec8047b564127b4249840e00c939b)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:36.076550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:36.132579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798ec8047b564127b4249840e00c939b'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.133254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798ec8047b564127b4249840e00c939b'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.133852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798ec8047b564127b4249840e00c939b'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.142610 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798ec8047b564127b4249840e00c939b'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.150477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798ec8047b564127b4249840e00c939b'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.169104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1249/2510] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:35 2022] DELETE /identity/v3/users/798ec8047b564127b4249840e00c939b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:36.362545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:36.373478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6766e-849d-4db2-9bfc-0984164dafbc service placement] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:36.430801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:36.438077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:36.438712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:36.438987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:36.439254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] PATH_INFO: `/v3/projects/4fd13262be8b4d1e9c660452ec33a983` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:36.440400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd13262be8b4d1e9c660452ec33a983)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:36.442311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:36.501242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd13262be8b4d1e9c660452ec33a983', 'request_id': 'req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.507089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd13262be8b4d1e9c660452ec33a983', 'request_id': 'req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.513130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd13262be8b4d1e9c660452ec33a983', 'request_id': 'req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.529550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] The token cache is being invalidate because project 4fd13262be8b4d1e9c660452ec33a983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:36.529837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.530466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.532082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1262/2511] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:36 2022] DELETE /identity/v3/projects/4fd13262be8b4d1e9c660452ec33a983 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:36.536247 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:36.537734 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:36.554137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b107fb-028f-40cd-bab1-2ed5fd7fa60f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:36.611411 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:36.612067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:36.612343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:36.612616 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] PATH_INFO: `/v3/users/a516aa46b3db48b89805f90f9ddd3f96` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:36.613763 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a516aa46b3db48b89805f90f9ddd3f96)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:36.615717 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:36.638492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:36.665913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a516aa46b3db48b89805f90f9ddd3f96'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.666776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a516aa46b3db48b89805f90f9ddd3f96'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.667390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a516aa46b3db48b89805f90f9ddd3f96'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.675975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a516aa46b3db48b89805f90f9ddd3f96'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.684366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-19ffa160-416a-4b44-bc05-5a6472958aca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a516aa46b3db48b89805f90f9ddd3f96'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:36.703409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1250/2512] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:36 2022] DELETE /identity/v3/users/a516aa46b3db48b89805f90f9ddd3f96 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:46:36.888182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:36.890628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:36.904916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67471c4a-1ba8-40fd-bdfa-fc2cc7d26ea3 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:36.966197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:36.966858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:36.967139 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:36.967409 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] PATH_INFO: `/v3/projects/d0cf90a269cb40d4bf975a0ce855f4f1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:36.968570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0cf90a269cb40d4bf975a0ce855f4f1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:36.971430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.025581 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0cf90a269cb40d4bf975a0ce855f4f1', 'request_id': 'req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:37.031995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0cf90a269cb40d4bf975a0ce855f4f1', 'request_id': 'req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:37.038028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0cf90a269cb40d4bf975a0ce855f4f1', 'request_id': 'req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:37.051848 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] The token cache is being invalidate because project d0cf90a269cb40d4bf975a0ce855f4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:37.052135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:37.052761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb79ff08-98b0-44a1-bb2e-2d66a342df5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:37.054247 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1263/2513] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:36 2022] DELETE /identity/v3/projects/d0cf90a269cb40d4bf975a0ce855f4f1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:37.059259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba2606-5200-41e0-b074-27609db8f725 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.059813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba2606-5200-41e0-b074-27609db8f725 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.059813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba2606-5200-41e0-b074-27609db8f725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.080325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-1cba2606-5200-41e0-b074-27609db8f725 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:37.081519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-1cba2606-5200-41e0-b074-27609db8f725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:37.143840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1251/2514] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:46:37.210790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390616b3-8413-48ca-b2aa-5ff420ae3510 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.211079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390616b3-8413-48ca-b2aa-5ff420ae3510 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.211445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390616b3-8413-48ca-b2aa-5ff420ae3510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.236655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-390616b3-8413-48ca-b2aa-5ff420ae3510 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:37.237848 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-390616b3-8413-48ca-b2aa-5ff420ae3510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:37.246546 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1264/2515] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:37.251046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba2606-5200-41e0-b074-27609db8f725 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.261514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba2606-5200-41e0-b074-27609db8f725 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.267481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.268113 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.268387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.268654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.269304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:46:37.269592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.271202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.283989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1252/2516] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:46:37.290322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390616b3-8413-48ca-b2aa-5ff420ae3510 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.301907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390616b3-8413-48ca-b2aa-5ff420ae3510 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.307352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.307982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.308256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.308523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.309142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.311052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.317500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:37.349204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1265/2517] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:37 2022] POST /identity/v3/projects => generated 361 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:37.353997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.366722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2bc5-5e00-43a7-8b7b-dea0f7876863 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.431185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.431875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.432200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.432496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.433126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.435050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.459093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1253/2518] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:37 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:37.463980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.475024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb8c2f-8023-4920-8582-c8bfc598aa44 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.480404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.481029 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.481303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.481571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.482216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.483857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.495242 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1266/2519] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:46:37.499326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.510173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c943-078f-49d1-8873-fab88bb5d1a9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.515661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59892-e684-495c-a9cf-da6e0752cabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.516328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59892-e684-495c-a9cf-da6e0752cabc admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.516624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59892-e684-495c-a9cf-da6e0752cabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.516875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59892-e684-495c-a9cf-da6e0752cabc admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.517507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59892-e684-495c-a9cf-da6e0752cabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.519092 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59892-e684-495c-a9cf-da6e0752cabc admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.529400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1254/2520] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:37.535754 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.550121 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e421f6-7eed-4025-a5df-fadbd534229e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.555975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.556627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.556937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.557210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] PATH_INFO: `/v3/projects/f47004826cfe4a1283ed7c1ec1579cd9/users/14bd257af54b4833ba004efc399312e8/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.574678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47004826cfe4a1283ed7c1ec1579cd9, user_id=14bd257af54b4833ba004efc399312e8, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.577427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.594551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1267/2521] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:37 2022] PUT /identity/v3/projects/f47004826cfe4a1283ed7c1ec1579cd9/users/14bd257af54b4833ba004efc399312e8/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:37.609657 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f67f8-619a-4f26-a1d0-d1c736c34f47 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.611342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f67f8-619a-4f26-a1d0-d1c736c34f47 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.611635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f67f8-619a-4f26-a1d0-d1c736c34f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.621420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:37.650782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-fb0f67f8-619a-4f26-a1d0-d1c736c34f47 None None] MFA Rules not processed for user `14bd257af54b4833ba004efc399312e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:37.652043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fb0f67f8-619a-4f26-a1d0-d1c736c34f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:37.727462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1255/2522] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:37.763653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.765427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:37.829636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461ff10-08e3-4cd4-a212-e58cc4f25996 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.836324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae11e5-4d10-4b05-a1f8-bbb793f0b6e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.837181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae11e5-4d10-4b05-a1f8-bbb793f0b6e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.837553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae11e5-4d10-4b05-a1f8-bbb793f0b6e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.837842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae11e5-4d10-4b05-a1f8-bbb793f0b6e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.848191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae11e5-4d10-4b05-a1f8-bbb793f0b6e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.849647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae11e5-4d10-4b05-a1f8-bbb793f0b6e4 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.858598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1268/2523] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:37.874108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f67f8-619a-4f26-a1d0-d1c736c34f47 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:37.940631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f67f8-619a-4f26-a1d0-d1c736c34f47 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:37.946008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:37.946654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:37.946932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:37.947205 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] PATH_INFO: `/v3/users/14bd257af54b4833ba004efc399312e8` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:37.948410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] RBAC: Authorizing `identity:delete_user(user_id=14bd257af54b4833ba004efc399312e8)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:37.950347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:37.998095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14bd257af54b4833ba004efc399312e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:37.998758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14bd257af54b4833ba004efc399312e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:37.999388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14bd257af54b4833ba004efc399312e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.007976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14bd257af54b4833ba004efc399312e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.014709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14bd257af54b4833ba004efc399312e8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.033543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1256/2524] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:37 2022] DELETE /identity/v3/users/14bd257af54b4833ba004efc399312e8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:38.038291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.038577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.038847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.058811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:38.060089 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:38.130557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1269/2525] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:38.140979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:38.142548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:38.161838 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:38.225787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f2d93-3255-466a-a00f-8c1323e2d20c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:38.229775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25af4bc-0575-48ad-a85b-e5552aaf137f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:38.230364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25af4bc-0575-48ad-a85b-e5552aaf137f service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.230634 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25af4bc-0575-48ad-a85b-e5552aaf137f service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.230900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25af4bc-0575-48ad-a85b-e5552aaf137f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.240086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25af4bc-0575-48ad-a85b-e5552aaf137f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:38.241531 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25af4bc-0575-48ad-a85b-e5552aaf137f service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:38.251006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1257/2526] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:38.542714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:38.544194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:38.558779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c74f02-a510-46ce-9b93-4e0eb605cf0a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:38.563945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:38.564833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.565098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.565362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] PATH_INFO: `/v3/projects/f47004826cfe4a1283ed7c1ec1579cd9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.566785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47004826cfe4a1283ed7c1ec1579cd9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:38.568958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:38.618636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47004826cfe4a1283ed7c1ec1579cd9', 'request_id': 'req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.619880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:38.626234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47004826cfe4a1283ed7c1ec1579cd9', 'request_id': 'req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.634303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47004826cfe4a1283ed7c1ec1579cd9', 'request_id': 'req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.649182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] The token cache is being invalidate because project f47004826cfe4a1283ed7c1ec1579cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:38.649468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.650074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-090c285c-3c33-4cd5-9f4a-c7fd02043b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:38.651565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1270/2527] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:38 2022] DELETE /identity/v3/projects/f47004826cfe4a1283ed7c1ec1579cd9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:38.656521 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc914c-ab1d-472c-8cc3-510f8f4e67f9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.656809 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc914c-ab1d-472c-8cc3-510f8f4e67f9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.657085 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc914c-ab1d-472c-8cc3-510f8f4e67f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.680580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f0cc914c-ab1d-472c-8cc3-510f8f4e67f9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:38.682373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f0cc914c-ab1d-472c-8cc3-510f8f4e67f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:38.756984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1258/2528] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:38.764846 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae368ce0-7487-45b1-bc98-ea2e540f9058 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.765131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae368ce0-7487-45b1-bc98-ea2e540f9058 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.765399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae368ce0-7487-45b1-bc98-ea2e540f9058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.785278 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-ae368ce0-7487-45b1-bc98-ea2e540f9058 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:38.786387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-ae368ce0-7487-45b1-bc98-ea2e540f9058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:38.794972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1271/2529] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:38.799599 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc914c-ab1d-472c-8cc3-510f8f4e67f9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:38.809962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc914c-ab1d-472c-8cc3-510f8f4e67f9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:38.815294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:38.815902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.816179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.816476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.817125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:46:38.817426 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:38.819026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:38.831988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1259/2530] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:46:38.836627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae368ce0-7487-45b1-bc98-ea2e540f9058 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:38.847259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae368ce0-7487-45b1-bc98-ea2e540f9058 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:38.852615 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:38.853220 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.853501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.853775 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.854392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:38.856369 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:38.886096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1272/2531] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:38 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:38.890467 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:38.900865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacee36-86b4-4bf8-baf3-53259c13dac4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:38.963942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:38.964575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:38.964850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:38.965117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:38.965766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:38.967718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:38.992118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1260/2532] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:38 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:38.996449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.006902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ea5c1-d56c-4f80-bdff-96f888ff21ec admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.011795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.012403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.012663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.012947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.013591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.015198 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.025591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1273/2533] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:39.029785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.040353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1256d-723e-4341-b789-a3d90d3b7ee7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.045767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e170b45-7c88-4dd6-967b-6fb3e28cb485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.046376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e170b45-7c88-4dd6-967b-6fb3e28cb485 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.046655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e170b45-7c88-4dd6-967b-6fb3e28cb485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.046946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e170b45-7c88-4dd6-967b-6fb3e28cb485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.047564 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e170b45-7c88-4dd6-967b-6fb3e28cb485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.049301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e170b45-7c88-4dd6-967b-6fb3e28cb485 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.060301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1261/2534] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:39.064415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.074394 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1a23e-ec7b-4e36-b3e5-e66331581eae admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.079301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.080065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.080341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.080619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] PATH_INFO: `/v3/projects/af1c243a5a284a79b700a4eac6cc467e/users/07aa652f4f3f4fd8b0f9dd259d27589b/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.095332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1c243a5a284a79b700a4eac6cc467e, user_id=07aa652f4f3f4fd8b0f9dd259d27589b, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.098001 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.112402 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1274/2535] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:39 2022] PUT /identity/v3/projects/af1c243a5a284a79b700a4eac6cc467e/users/07aa652f4f3f4fd8b0f9dd259d27589b/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:39.124764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035bf73-b78f-47c2-a3bd-31f77e526244 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.125149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035bf73-b78f-47c2-a3bd-31f77e526244 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.125540 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035bf73-b78f-47c2-a3bd-31f77e526244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.133833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:39.157225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5035bf73-b78f-47c2-a3bd-31f77e526244 None None] MFA Rules not processed for user `07aa652f4f3f4fd8b0f9dd259d27589b`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:39.158523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5035bf73-b78f-47c2-a3bd-31f77e526244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:39.225035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1262/2536] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:39.229721 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.299960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78459a9b-bd85-469c-8ad5-c08a7d4c7055 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.307160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.307799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.308080 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.308352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.308988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.310947 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.341758 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1275/2537] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:39 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:39.346038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035bf73-b78f-47c2-a3bd-31f77e526244 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.357219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035bf73-b78f-47c2-a3bd-31f77e526244 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.425650 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.426259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.426532 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.426781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.427458 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.429367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.453067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1263/2538] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:39 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:46:39.458414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.469090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685566ed-fb5b-4b86-9fd4-0dd2e404deb7 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.474219 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.474815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.475096 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.475360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.476034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.477648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.488630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1276/2539] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:39.492932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.503812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74855cf7-891c-4fe4-bed9-33a6be9dd724 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.509186 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.509870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.510141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.510408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] PATH_INFO: `/v3/projects/0f8883f0c80c45abb8b64bd06b39634b/users/0dfb2549b3a8462dbb8323ea5ffaa125/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.525504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8883f0c80c45abb8b64bd06b39634b, user_id=0dfb2549b3a8462dbb8323ea5ffaa125, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.528178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.542768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1264/2540] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:39 2022] PUT /identity/v3/projects/0f8883f0c80c45abb8b64bd06b39634b/users/0dfb2549b3a8462dbb8323ea5ffaa125/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:39.546884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.556982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1238d-a48c-4c2c-8726-0a3caa7c5253 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.604125 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:39.619135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c57e7-9d6d-4e3d-bc3c-8b11b167f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.620520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c57e7-9d6d-4e3d-bc3c-8b11b167f309 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.620638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c57e7-9d6d-4e3d-bc3c-8b11b167f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.620837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c57e7-9d6d-4e3d-bc3c-8b11b167f309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.621486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c57e7-9d6d-4e3d-bc3c-8b11b167f309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.623189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c57e7-9d6d-4e3d-bc3c-8b11b167f309 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.633274 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1277/2541] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:39.637670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.648314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08cd00-9b3e-4030-a6bd-0bcdfba57d3d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.653457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.654150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.654422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.654685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] PATH_INFO: `/v3/projects/0f8883f0c80c45abb8b64bd06b39634b/users/0dfb2549b3a8462dbb8323ea5ffaa125/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.656870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8883f0c80c45abb8b64bd06b39634b, user_id=0dfb2549b3a8462dbb8323ea5ffaa125, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.659608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.670859 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1265/2542] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:39 2022] PUT /identity/v3/projects/0f8883f0c80c45abb8b64bd06b39634b/users/0dfb2549b3a8462dbb8323ea5ffaa125/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:46:39.681403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cf02c-4b97-491a-b046-f8756f7f3b83 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.681689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cf02c-4b97-491a-b046-f8756f7f3b83 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.681955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cf02c-4b97-491a-b046-f8756f7f3b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.714307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f82cf02c-4b97-491a-b046-f8756f7f3b83 None None] MFA Rules not processed for user `0dfb2549b3a8462dbb8323ea5ffaa125`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:39.715557 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f82cf02c-4b97-491a-b046-f8756f7f3b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:39.784952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1278/2543] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:39.796924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:39.799543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:39.848329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:39.859971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a783345-4e35-4bc3-80c1-1a2ef12074bf admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:39.864250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1c40d-1a05-48f9-b589-2f0ae2b7d546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:39.864855 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1c40d-1a05-48f9-b589-2f0ae2b7d546 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:39.865131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1c40d-1a05-48f9-b589-2f0ae2b7d546 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:39.865420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1c40d-1a05-48f9-b589-2f0ae2b7d546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:39.929606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1c40d-1a05-48f9-b589-2f0ae2b7d546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:39.931128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1c40d-1a05-48f9-b589-2f0ae2b7d546 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:39.940874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1266/2544] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:42.548621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cf02c-4b97-491a-b046-f8756f7f3b83 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:42.626064 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cf02c-4b97-491a-b046-f8756f7f3b83 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:42.632957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:42.633592 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:42.633861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:42.634126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] PATH_INFO: `/v3/users/07aa652f4f3f4fd8b0f9dd259d27589b` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:42.635387 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07aa652f4f3f4fd8b0f9dd259d27589b)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:42.637348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:42.690383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07aa652f4f3f4fd8b0f9dd259d27589b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:42.691466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07aa652f4f3f4fd8b0f9dd259d27589b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:42.698571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07aa652f4f3f4fd8b0f9dd259d27589b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:42.699211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07aa652f4f3f4fd8b0f9dd259d27589b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:42.707987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07aa652f4f3f4fd8b0f9dd259d27589b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:42.715421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:42.729832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1279/2545] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:42 2022] DELETE /identity/v3/users/07aa652f4f3f4fd8b0f9dd259d27589b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:42.734485 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:42.735012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:42.735111 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:42.755954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:42.757181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:42.817662 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1267/2546] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:42.828083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:42.829710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:42.900916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc366b3-4f1e-48b3-bdf7-a1a870039274 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:42.905353 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:42.905973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:42.906254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:42.906527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:42.917458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:42.918908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:42.929364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1280/2547] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:43.204754 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:43.206377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:43.219025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b226118-4316-4fb6-ad42-f472af48ec55 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:43.224691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:43.225383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:43.225682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:43.225954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] PATH_INFO: `/v3/projects/af1c243a5a284a79b700a4eac6cc467e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:43.227066 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1c243a5a284a79b700a4eac6cc467e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:43.229013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:43.277813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1c243a5a284a79b700a4eac6cc467e', 'request_id': 'req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.283194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1c243a5a284a79b700a4eac6cc467e', 'request_id': 'req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.292151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1c243a5a284a79b700a4eac6cc467e', 'request_id': 'req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.310514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] The token cache is being invalidate because project af1c243a5a284a79b700a4eac6cc467e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:43.310804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.311580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.313251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1268/2548] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:43 2022] DELETE /identity/v3/projects/af1c243a5a284a79b700a4eac6cc467e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:43.317569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:43.327529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84cfba-1b73-4af9-86d0-2b6386f79f14 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:43.385744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:43.386367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:43.386642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:43.386903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] PATH_INFO: `/v3/users/0dfb2549b3a8462dbb8323ea5ffaa125` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:43.388088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dfb2549b3a8462dbb8323ea5ffaa125)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:43.390011 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:43.443151 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dfb2549b3a8462dbb8323ea5ffaa125'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.443815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dfb2549b3a8462dbb8323ea5ffaa125'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.450520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dfb2549b3a8462dbb8323ea5ffaa125'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.451135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dfb2549b3a8462dbb8323ea5ffaa125'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.459475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-20374736-8692-44c9-8226-4bac562c5c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dfb2549b3a8462dbb8323ea5ffaa125'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.477212 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1281/2549] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:43 2022] DELETE /identity/v3/users/0dfb2549b3a8462dbb8323ea5ffaa125 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:43.767374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:43.769081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:43.783558 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b56e1-8e56-408e-a7c0-d3b0a36498b5 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:43.802772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:43.844164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:43.844721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:43.844995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:43.845265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] PATH_INFO: `/v3/projects/0f8883f0c80c45abb8b64bd06b39634b` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:43.846391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8883f0c80c45abb8b64bd06b39634b)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:43.848419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:43.896060 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8883f0c80c45abb8b64bd06b39634b', 'request_id': 'req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.902125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8883f0c80c45abb8b64bd06b39634b', 'request_id': 'req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.908988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8883f0c80c45abb8b64bd06b39634b', 'request_id': 'req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.924962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] The token cache is being invalidate because project 0f8883f0c80c45abb8b64bd06b39634b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:43.925253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.925947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-1724b0b3-698c-4c98-a013-0ecf7e7a47e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:43.927461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1269/2550] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:43 2022] DELETE /identity/v3/projects/0f8883f0c80c45abb8b64bd06b39634b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:43.932881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a4ab-1885-41d4-8e41-ba58da26de28 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:43.933378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a4ab-1885-41d4-8e41-ba58da26de28 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:43.933478 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a4ab-1885-41d4-8e41-ba58da26de28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:43.953315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-46b5a4ab-1885-41d4-8e41-ba58da26de28 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:43.954540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-46b5a4ab-1885-41d4-8e41-ba58da26de28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:43.960221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:44.017540 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1282/2551] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:44.026319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc031e-b8cb-4c94-b1b9-7f36b936b9a9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.026572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc031e-b8cb-4c94-b1b9-7f36b936b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.026868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc031e-b8cb-4c94-b1b9-7f36b936b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.048442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-b9fc031e-b8cb-4c94-b1b9-7f36b936b9a9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:44.049652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b9fc031e-b8cb-4c94-b1b9-7f36b936b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:44.058834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1270/2552] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:44.063189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a4ab-1885-41d4-8e41-ba58da26de28 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:44.073422 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a4ab-1885-41d4-8e41-ba58da26de28 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:44.078832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:44.079447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.079761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.080042 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.080687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:46:44.080991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:44.082636 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:44.095111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1283/2553] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:46:44.099516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc031e-b8cb-4c94-b1b9-7f36b936b9a9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:44.109271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc031e-b8cb-4c94-b1b9-7f36b936b9a9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:44.114553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:44.115145 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.115400 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.115718 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.116388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:44.118277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:44.148802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1271/2554] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:46:44 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:44.153055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:44.163736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409b8d-127b-4953-a65c-7508c75b0dab admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:44.223031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:44.223660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.223943 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.224216 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.224840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:44.226788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:44.248524 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1284/2555] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:46:44 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:46:44.253970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:44.264672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea0013-4f9c-4fb1-a606-49ea4de1c109 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:44.269889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:44.270491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.270772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.271040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.271723 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:44.273425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:44.283692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1272/2556] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:44.288323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:44.302185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c86b4a-e5a2-4089-91c5-756017201a1a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:44.307679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b69e7c-0e35-43f3-96de-2d92b665f92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:44.308286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b69e7c-0e35-43f3-96de-2d92b665f92c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.308587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b69e7c-0e35-43f3-96de-2d92b665f92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.308854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b69e7c-0e35-43f3-96de-2d92b665f92c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.309493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b69e7c-0e35-43f3-96de-2d92b665f92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:44.311110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b69e7c-0e35-43f3-96de-2d92b665f92c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:44.322981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1285/2557] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:46:44.327284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:44.338389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00325ce-9fdd-480d-987a-9bb0c6b077c7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:44.343621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:44.344299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.344572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.344843 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] PATH_INFO: `/v3/projects/db82dbc77f524d47bf23ebbbc4a114bf/users/3495ac82564f45e4945338499671fee8/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.360580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db82dbc77f524d47bf23ebbbc4a114bf, user_id=3495ac82564f45e4945338499671fee8, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:44.363252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:44.377942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1273/2558] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:46:44 2022] PUT /identity/v3/projects/db82dbc77f524d47bf23ebbbc4a114bf/users/3495ac82564f45e4945338499671fee8/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:44.389498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fafb9-8a5f-449f-bceb-33a48bc7856f None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.389838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fafb9-8a5f-449f-bceb-33a48bc7856f None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.390101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fafb9-8a5f-449f-bceb-33a48bc7856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.419954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-766fafb9-8a5f-449f-bceb-33a48bc7856f None None] MFA Rules not processed for user `3495ac82564f45e4945338499671fee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:44.421168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-766fafb9-8a5f-449f-bceb-33a48bc7856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:44.490723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1286/2559] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:44.505990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:44.508208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:44.564694 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:44.591898 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b131b4-9b9a-4386-b11d-82472af60ab5 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:44.595907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5fd99-7514-4997-9202-2e04e9e7cf1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:44.596516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fd99-7514-4997-9202-2e04e9e7cf1f service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:44.596790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fd99-7514-4997-9202-2e04e9e7cf1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:44.597053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5fd99-7514-4997-9202-2e04e9e7cf1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:44.605359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5fd99-7514-4997-9202-2e04e9e7cf1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:44.606774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5fd99-7514-4997-9202-2e04e9e7cf1f service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:44.615655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1274/2560] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:52.811909 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fafb9-8a5f-449f-bceb-33a48bc7856f None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:52.821566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:52.879559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fafb9-8a5f-449f-bceb-33a48bc7856f None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:52.884582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:52.885230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:52.885510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:52.885778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] PATH_INFO: `/v3/users/3495ac82564f45e4945338499671fee8` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:52.886957 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3495ac82564f45e4945338499671fee8)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:52.888922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:52.951043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3495ac82564f45e4945338499671fee8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:52.951801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3495ac82564f45e4945338499671fee8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:52.958748 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3495ac82564f45e4945338499671fee8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:52.959373 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3495ac82564f45e4945338499671fee8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:52.968223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3495ac82564f45e4945338499671fee8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:52.985741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1287/2561] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:46:52 2022] DELETE /identity/v3/users/3495ac82564f45e4945338499671fee8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:52.992795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:52.993297 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:52.993386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:53.017383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:46:53.018583 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:53.089047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1275/2562] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:46:53.099062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:53.100606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:53.160934 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5f9c7-70e0-4fbf-8fbf-dc5f51046637 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:53.165236 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:53.167425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:53.167425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:53.167425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:53.176620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:53.178068 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:53.187936 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1288/2563] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:46:53.477578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:53.479727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:53.494131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c64fa4-34e3-44ca-9067-a05b9952da8d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:53.500169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:53.500818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:53.501091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:53.501356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] PATH_INFO: `/v3/projects/db82dbc77f524d47bf23ebbbc4a114bf` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:53.502619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db82dbc77f524d47bf23ebbbc4a114bf)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:53.504830 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:53.558732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db82dbc77f524d47bf23ebbbc4a114bf', 'request_id': 'req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:53.564465 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db82dbc77f524d47bf23ebbbc4a114bf', 'request_id': 'req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:53.570792 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db82dbc77f524d47bf23ebbbc4a114bf', 'request_id': 'req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:53.585437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] The token cache is being invalidate because project db82dbc77f524d47bf23ebbbc4a114bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:46:53.586699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:53.587372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:46:53.589864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1276/2564] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:46:53 2022] DELETE /identity/v3/projects/db82dbc77f524d47bf23ebbbc4a114bf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:46:59.010340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:46:59.020688 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156968c-70d7-4ad9-9c7b-c0ba90bfd1e8 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:46:59.048226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:46:59.073412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:46:59.074021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:46:59.074300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:46:59.074571 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:46:59.076421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:46:59.135196 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:46:59.136631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:46:59.145731 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1289/2565] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:46:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:47:03.582250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:03.941744 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:03.943395 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:03.955524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff0388-0bfb-44f6-a3e2-3147b57f2725 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:03.958999 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:03.959612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:03.959888 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:03.960155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:03.961786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:04.027131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:04.028609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:04.038687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1277/2566] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:47:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:47:07.485124 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:07.486714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:07.581479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fcb92-7aee-4848-936b-abac6c8a3e01 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:07.586498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:07.587113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:07.587393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:07.587691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:07.590074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:07.603733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:07.605715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:07.615743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1290/2567] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:47:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:47:10.489304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:10.490912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:10.503527 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9812f4-27e9-459e-a6cd-930c3d7994ec service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:10.509110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:10.509751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:10.510026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:10.510294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] PATH_INFO: `/v3/users/f4010e99c30b4a49b19c6b8fa2b5f7f2` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:10.511667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4010e99c30b4a49b19c6b8fa2b5f7f2)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:10.513587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:10.563041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4010e99c30b4a49b19c6b8fa2b5f7f2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.563722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4010e99c30b4a49b19c6b8fa2b5f7f2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.564318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4010e99c30b4a49b19c6b8fa2b5f7f2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.572984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4010e99c30b4a49b19c6b8fa2b5f7f2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.582849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4010e99c30b4a49b19c6b8fa2b5f7f2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.602115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1278/2568] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:47:10 2022] DELETE /identity/v3/users/f4010e99c30b4a49b19c6b8fa2b5f7f2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:47:10.784971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:10.798093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e434a99-3549-46c4-956f-2d661ae8bf68 service placement] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:10.857462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:10.858083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:10.858381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:10.858656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] PATH_INFO: `/v3/projects/638a0c07a94f43d4807931639fd6a93e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:10.859842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=638a0c07a94f43d4807931639fd6a93e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:10.861743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:10.879486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:10.913447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '638a0c07a94f43d4807931639fd6a93e', 'request_id': 'req-7dd0b19e-049f-4db6-818f-7547cc1bba2e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.919004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '638a0c07a94f43d4807931639fd6a93e', 'request_id': 'req-7dd0b19e-049f-4db6-818f-7547cc1bba2e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.925170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '638a0c07a94f43d4807931639fd6a93e', 'request_id': 'req-7dd0b19e-049f-4db6-818f-7547cc1bba2e'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.940330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] The token cache is being invalidate because project 638a0c07a94f43d4807931639fd6a93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:47:10.940639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.941318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:10.942911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1291/2569] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:47:10 2022] DELETE /identity/v3/projects/638a0c07a94f43d4807931639fd6a93e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:47:10.947079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:10.948651 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:10.962451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ba3e4-3150-4293-ac9d-bbcd6f695be5 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:11.024271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:11.025231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:11.025700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:11.026142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] PATH_INFO: `/v3/users/127f43b053ca42d1a1683f8a8ad5c835` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:11.027956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] RBAC: Authorizing `identity:delete_user(user_id=127f43b053ca42d1a1683f8a8ad5c835)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:11.030074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:11.039507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:11.096790 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127f43b053ca42d1a1683f8a8ad5c835'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.097606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127f43b053ca42d1a1683f8a8ad5c835'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.098384 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127f43b053ca42d1a1683f8a8ad5c835'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.107130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127f43b053ca42d1a1683f8a8ad5c835'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.114336 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127f43b053ca42d1a1683f8a8ad5c835'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.132019 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1279/2570] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:47:10 2022] DELETE /identity/v3/users/127f43b053ca42d1a1683f8a8ad5c835 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:47:11.312037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:11.313583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:11.327820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0b19e-049f-4db6-818f-7547cc1bba2e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:11.387383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:11.388053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:11.388329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:11.388598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] PATH_INFO: `/v3/projects/c2b4b850f6544bc4bdeddfbce846c3ee` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:11.389787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b4b850f6544bc4bdeddfbce846c3ee)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:11.391713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:11.448685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b4b850f6544bc4bdeddfbce846c3ee', 'request_id': 'req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.455641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b4b850f6544bc4bdeddfbce846c3ee', 'request_id': 'req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.462680 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b4b850f6544bc4bdeddfbce846c3ee', 'request_id': 'req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.490907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] The token cache is being invalidate because project c2b4b850f6544bc4bdeddfbce846c3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:47:11.491856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.492482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0dd67c17-cc7a-44ff-ace2-d8ba28ececde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:11.494004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1292/2571] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:47:11 2022] DELETE /identity/v3/projects/c2b4b850f6544bc4bdeddfbce846c3ee => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:47:21.128520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:21.486347 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=69,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:34.396345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:34.398439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:34.471285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ee6a-9ed8-42e0-bc30-95cb4928363e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:34.524484 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:34.525112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:34.525391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:34.525658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:34.527387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:34.597040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:34.598421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:34.609061 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1280/2572] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:47:35.298944 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:35.299446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:35.299548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:35.331370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:35.332604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:35.397477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1293/2573] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:47:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:35.453405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:35.454845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:35.530039 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d7b1b-f9d7-4ca6-b82d-a2922b0ce4e3 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:35.535673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:35.536286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:35.536561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:35.536828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:35.544463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:35.546749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:35.547957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:35.557623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1281/2574] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:47:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:47:38.917746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:38.919876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:38.941429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9875993-fb12-4fd9-b80a-fa940621b348 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:38.947718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:38.948376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:38.948654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:38.948924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] PATH_INFO: `/v3/users/8e68f520aaff495abfb4b7a5271936e1` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:38.951147 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e68f520aaff495abfb4b7a5271936e1)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:38.953279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:39.015932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e68f520aaff495abfb4b7a5271936e1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.017009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e68f520aaff495abfb4b7a5271936e1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.025607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e68f520aaff495abfb4b7a5271936e1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.027077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e68f520aaff495abfb4b7a5271936e1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.038390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-0615a4c5-9f9f-4b60-a89a-e9b73d578c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e68f520aaff495abfb4b7a5271936e1'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.063620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1294/2575] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:47:38 2022] DELETE /identity/v3/users/8e68f520aaff495abfb4b7a5271936e1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:47:39.259179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:39.279536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb33b38-e308-4d7c-be33-2265147a1ed0 service swift] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:39.370767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:39.371433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.371742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.372024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] PATH_INFO: `/v3/projects/8e8481f1683045d4a63b3d33e0dab620` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.373538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8481f1683045d4a63b3d33e0dab620)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:39.376279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:39.433897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8481f1683045d4a63b3d33e0dab620', 'request_id': 'req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.442220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8481f1683045d4a63b3d33e0dab620', 'request_id': 'req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.449575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8481f1683045d4a63b3d33e0dab620', 'request_id': 'req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.469559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] The token cache is being invalidate because project 8e8481f1683045d4a63b3d33e0dab620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:47:39.470182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.471055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-9fa69e8e-110c-48ec-98f5-30bdfb74bacd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:47:39.476864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1282/2576] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:47:39 2022] DELETE /identity/v3/projects/8e8481f1683045d4a63b3d33e0dab620 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:47:39.478564 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae7e9-dbf0-4523-9644-a69204958f01 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.478854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae7e9-dbf0-4523-9644-a69204958f01 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.479133 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae7e9-dbf0-4523-9644-a69204958f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.504277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-fa8ae7e9-dbf0-4523-9644-a69204958f01 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:39.505700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fa8ae7e9-dbf0-4523-9644-a69204958f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:39.607932 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1295/2577] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:39.609829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343617d3-454a-42b1-ac9a-a31217a8e357 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.610239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343617d3-454a-42b1-ac9a-a31217a8e357 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.610574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343617d3-454a-42b1-ac9a-a31217a8e357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.637514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-343617d3-454a-42b1-ac9a-a31217a8e357 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:39.639934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-343617d3-454a-42b1-ac9a-a31217a8e357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:39.649195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1283/2578] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:39.653552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ae7e9-dbf0-4523-9644-a69204958f01 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:39.658686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:39.666400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ae7e9-dbf0-4523-9644-a69204958f01 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:39.676453 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:39.677332 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.677670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.677946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.678610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:47:39.678911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:39.680583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:39.704237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343617d3-454a-42b1-ac9a-a31217a8e357 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:39.709090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1296/2579] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:47:39.721029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343617d3-454a-42b1-ac9a-a31217a8e357 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:39.730037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:39.730665 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.730944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.731214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.731896 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:39.733819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:39.785586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1284/2580] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:47:39 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:47:39.792173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:39.803886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4963e-37df-4728-99ea-b87aef0fe0ab admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:39.881790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:39.882558 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.882925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.883263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.884040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:39.886087 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:39.918727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1297/2581] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:47:39 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:47:39.922783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:39.934742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94df65-6435-42cf-b1e9-1714feb1466d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:39.939882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:39.940516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.940911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.941189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.942100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:39.944710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:39.960470 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1285/2582] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:47:39.961602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:39.973935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89343e89-62eb-4335-a3a7-9739a89e1635 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:39.980374 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298a493-7b1b-480d-a5d9-81cd4b3f2d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:39.981000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a493-7b1b-480d-a5d9-81cd4b3f2d47 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:39.981287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a493-7b1b-480d-a5d9-81cd4b3f2d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:39.981555 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a493-7b1b-480d-a5d9-81cd4b3f2d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:39.982837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298a493-7b1b-480d-a5d9-81cd4b3f2d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:39.984390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298a493-7b1b-480d-a5d9-81cd4b3f2d47 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:39.996525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1298/2583] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:47:40.001435 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:40.012714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d244dec-a32d-4817-a1eb-40b177d2ef01 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:40.019767 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:40.020675 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:40.021120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:40.021455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] PATH_INFO: `/v3/projects/fae948b0baa04f2eb07b2aee2785dcdd/users/ed76a574d2fc4bd8a1f1176d0fef86e2/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:40.039561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae948b0baa04f2eb07b2aee2785dcdd, user_id=ed76a574d2fc4bd8a1f1176d0fef86e2, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:40.043126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:40.058416 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1286/2584] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:47:40 2022] PUT /identity/v3/projects/fae948b0baa04f2eb07b2aee2785dcdd/users/ed76a574d2fc4bd8a1f1176d0fef86e2/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:47:40.063602 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ddf0-620a-45e5-a418-198a2be292d8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:40.064084 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ddf0-620a-45e5-a418-198a2be292d8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:40.064428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ddf0-620a-45e5-a418-198a2be292d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:40.084959 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-5410ddf0-620a-45e5-a418-198a2be292d8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:40.086217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5410ddf0-620a-45e5-a418-198a2be292d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:40.159621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1299/2585] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:40.169287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:40.171054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:40.202799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:40.241764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9253e8-5162-4ffd-9809-3a127f5c9c1a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:40.248906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:40.249633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:40.250007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:40.250366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:40.261359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:40.263891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:40.275595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1287/2586] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:47:45.265119 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c3df-6aa5-428f-b17b-a3ce5d14ea3d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:45.265621 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c3df-6aa5-428f-b17b-a3ce5d14ea3d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:45.265706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c3df-6aa5-428f-b17b-a3ce5d14ea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:45.297334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-78c2c3df-6aa5-428f-b17b-a3ce5d14ea3d None None] MFA Rules not processed for user `ed76a574d2fc4bd8a1f1176d0fef86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:45.299081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-78c2c3df-6aa5-428f-b17b-a3ce5d14ea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:45.386718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:45.399041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1300/2587] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:47:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:45.413639 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:45.415251 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:45.558080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7740f-2a76-4983-a0af-fc21ca8d84ec service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:45.564129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b122c-5048-445e-b898-e25c3eaad02a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:45.564765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b122c-5048-445e-b898-e25c3eaad02a service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:45.565114 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b122c-5048-445e-b898-e25c3eaad02a service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:45.565375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b122c-5048-445e-b898-e25c3eaad02a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:45.574258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b122c-5048-445e-b898-e25c3eaad02a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:45.575909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b122c-5048-445e-b898-e25c3eaad02a service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:45.586258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1288/2588] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:47:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:47:47.166854 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7c93-ccea-4228-9e4a-56bfa1304421 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:47.167367 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7c93-ccea-4228-9e4a-56bfa1304421 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:47.167469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7c93-ccea-4228-9e4a-56bfa1304421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:47.188616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3bfa7c93-ccea-4228-9e4a-56bfa1304421 None None] MFA Rules not processed for user `ed76a574d2fc4bd8a1f1176d0fef86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:47.190743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3bfa7c93-ccea-4228-9e4a-56bfa1304421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:47.201667 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1301/2589] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:47:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:50.919286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301534-d3a6-4c35-8b85-4f79f5a36919 None None] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:50.919772 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301534-d3a6-4c35-8b85-4f79f5a36919 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:50.920095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301534-d3a6-4c35-8b85-4f79f5a36919 None None] PATH_INFO: `/` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:50.921176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1289/2590] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 22 05:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 05:47:50.927324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fa820-683d-4ea0-afb3-29b6db50a17d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:50.927960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fa820-683d-4ea0-afb3-29b6db50a17d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:50.927960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fa820-683d-4ea0-afb3-29b6db50a17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:50.951411 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-850fa820-683d-4ea0-afb3-29b6db50a17d None None] MFA Rules not processed for user `a3b919bcbb104fa3bf11847b6a35a221`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:50.952654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-850fa820-683d-4ea0-afb3-29b6db50a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:50.964437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1302/2591] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 22 05:47:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:50.981541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301534-d3a6-4c35-8b85-4f79f5a36919 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:50.983874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-81301534-d3a6-4c35-8b85-4f79f5a36919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:51.072351 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301534-d3a6-4c35-8b85-4f79f5a36919 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:51.076305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:51.077087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:51.077429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:51.077766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:51.092169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:51.094008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:51.106834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1290/2592] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:47:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:47:54.905046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:54.906061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:54.906178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:54.926567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:47:54.928014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:54.991795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1303/2593] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:47:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:47:55.004497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:47:55.006733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:47:55.048334 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:47:55.085613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7731f-55de-4922-bab1-af8a59733ed2 service glance] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:47:55.091376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc442f-b3b3-4e42-8b00-f2126fd6916f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:47:55.092036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc442f-b3b3-4e42-8b00-f2126fd6916f service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:47:55.092307 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc442f-b3b3-4e42-8b00-f2126fd6916f service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:47:55.092584 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc442f-b3b3-4e42-8b00-f2126fd6916f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:47:55.101995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc442f-b3b3-4e42-8b00-f2126fd6916f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:47:55.106632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc442f-b3b3-4e42-8b00-f2126fd6916f service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:47:55.117857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1291/2594] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:48:04.984228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=32 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:48:05.112907 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=17 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:48:12.844381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:48:12.846529 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:48:12.859213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6980c8-380c-4779-b786-c688b9dc79d0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:48:12.863228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:48:12.863916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:48:12.864195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:48:12.864470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:48:12.866613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:48:12.881408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:48:12.883413 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:48:12.914838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1304/2595] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:48:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:48:13.550497 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c092049-7463-4632-9791-f702d47dc105 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:48:13.551004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c092049-7463-4632-9791-f702d47dc105 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:48:13.551004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c092049-7463-4632-9791-f702d47dc105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:48:13.572956 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0c092049-7463-4632-9791-f702d47dc105 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:48:13.574170 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0c092049-7463-4632-9791-f702d47dc105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:48:13.582742 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1292/2596] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 22 05:48:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:48:13.596018 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:48:13.607904 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8194fe-252a-46ee-8b05-9b85b7ba7c90 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:48:13.613668 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:48:13.614300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:48:13.614580 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:48:13.614856 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:48:13.623874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:48:13.625464 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:48:13.633869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1305/2597] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:48:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:48:14.108128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c092049-7463-4632-9791-f702d47dc105 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:48:14.118861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c092049-7463-4632-9791-f702d47dc105 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:48:14.122941 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:48:14.123543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:48:14.123854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:48:14.124119 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:48:14.125899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:48:14.141596 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:48:14.143184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:48:14.155110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1293/2598] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:48:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:48:23.630584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=12 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:48:24.149551 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=9 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:18.092420 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:18.102490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249440-aa30-4ec6-888e-cd3149cf2752 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:18.107533 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:18.108172 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:18.108448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:18.108718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:18.110844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:18.124149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:18.125593 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:18.136355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1306/2599] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:21.341345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:21.342821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:21.359845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cec10-fd2d-4e82-9cf7-852bcbbbedb3 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:21.368864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:21.369528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:21.369805 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:21.370077 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] PATH_INFO: `/v3/users/2864050fd1a64646a8760f0172be8f4a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:21.371255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2864050fd1a64646a8760f0172be8f4a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:21.373259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:21.422161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2864050fd1a64646a8760f0172be8f4a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.422808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2864050fd1a64646a8760f0172be8f4a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.423392 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2864050fd1a64646a8760f0172be8f4a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.431745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2864050fd1a64646a8760f0172be8f4a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.438816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-e899ccc7-882b-4088-b4b5-7cb0981bbeaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2864050fd1a64646a8760f0172be8f4a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.456414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1294/2600] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:21 2022] DELETE /identity/v3/users/2864050fd1a64646a8760f0172be8f4a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:49:21.631950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:21.643120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5cfc2-3ffe-4f7c-8d4e-9a0581f5f7c0 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:21.705476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:21.706122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:21.706392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:21.706661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] PATH_INFO: `/v3/projects/e27e3216211942959311372c2d078202` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:21.707820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27e3216211942959311372c2d078202)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:21.709732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:21.758078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27e3216211942959311372c2d078202', 'request_id': 'req-2c8191a7-1853-4a43-9815-42c39d4e08c8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.764106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27e3216211942959311372c2d078202', 'request_id': 'req-2c8191a7-1853-4a43-9815-42c39d4e08c8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.771163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27e3216211942959311372c2d078202', 'request_id': 'req-2c8191a7-1853-4a43-9815-42c39d4e08c8'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.785554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] The token cache is being invalidate because project e27e3216211942959311372c2d078202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:21.785832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.786447 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-2c8191a7-1853-4a43-9815-42c39d4e08c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:21.787998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1307/2601] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:21 2022] DELETE /identity/v3/projects/e27e3216211942959311372c2d078202 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:21.793281 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db659ec-5598-412f-a9aa-9fa4b8055bb7 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:21.793593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db659ec-5598-412f-a9aa-9fa4b8055bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:21.793869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db659ec-5598-412f-a9aa-9fa4b8055bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:21.813469 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9db659ec-5598-412f-a9aa-9fa4b8055bb7 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:21.814686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9db659ec-5598-412f-a9aa-9fa4b8055bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:21.878180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1295/2602] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:49:21.886658 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f83224-46ff-448a-ad29-81d9c5ede051 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:21.886991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f83224-46ff-448a-ad29-81d9c5ede051 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:21.887273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f83224-46ff-448a-ad29-81d9c5ede051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:21.910257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c2f83224-46ff-448a-ad29-81d9c5ede051 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:21.913079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c2f83224-46ff-448a-ad29-81d9c5ede051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:21.924251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1308/2603] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:21.928949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db659ec-5598-412f-a9aa-9fa4b8055bb7 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:21.939874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db659ec-5598-412f-a9aa-9fa4b8055bb7 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:21.946441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:21.947050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:21.947321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:21.947617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:21.948267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:49:21.948569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:21.950185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:21.965057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1296/2604] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:49:21.969735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f83224-46ff-448a-ad29-81d9c5ede051 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:21.980051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f83224-46ff-448a-ad29-81d9c5ede051 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:21.989054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:21.989689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:21.989969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:21.990237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:21.990872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:21.992827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.035855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1309/2605] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:21 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:22.040231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.051125 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a1f9c-17a4-45cf-98ed-ebc51f9c546d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.123529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.124163 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.124452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.124746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.125373 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.127314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.135329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:22.152228 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1297/2606] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:22 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:22.156550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.167016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e880a-a1d4-49a3-9935-cc5a08f4482e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.172304 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.172910 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.173187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.173457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.174098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.175774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.185995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1310/2607] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:22.189997 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.200440 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ce19a-35cd-45da-9328-2d99472f122b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.206366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df0b46-2d8e-4f19-b234-b5978449bf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.206961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0b46-2d8e-4f19-b234-b5978449bf5c admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.207234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0b46-2d8e-4f19-b234-b5978449bf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.207493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0b46-2d8e-4f19-b234-b5978449bf5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.208176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df0b46-2d8e-4f19-b234-b5978449bf5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.209771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df0b46-2d8e-4f19-b234-b5978449bf5c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.221499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1298/2608] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:22.225803 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.236963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02bbe7-a44a-4ca0-9c8c-5dd09a82cdfc admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.243303 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.243997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.244277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.244553 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] PATH_INFO: `/v3/projects/4370396362ea4362aa2fbedf8ec59ab3/users/7c490d7b5d334f5fa16d331ed452b944/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.261724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4370396362ea4362aa2fbedf8ec59ab3, user_id=7c490d7b5d334f5fa16d331ed452b944, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.264469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.284582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1311/2609] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:22 2022] PUT /identity/v3/projects/4370396362ea4362aa2fbedf8ec59ab3/users/7c490d7b5d334f5fa16d331ed452b944/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:22.300775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00dad-b2fd-4dc2-840e-cfd3fe8d72e9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.300955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00dad-b2fd-4dc2-840e-cfd3fe8d72e9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.301313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00dad-b2fd-4dc2-840e-cfd3fe8d72e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.337089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-bbf00dad-b2fd-4dc2-840e-cfd3fe8d72e9 None None] MFA Rules not processed for user `7c490d7b5d334f5fa16d331ed452b944`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:22.338591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bbf00dad-b2fd-4dc2-840e-cfd3fe8d72e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:22.411339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1299/2610] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:22.416223 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.422664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:22.490228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924be02-03cb-4d5d-a647-abef8a73b931 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.495432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.495882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.496181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.496450 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.497069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.498963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.528231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1312/2611] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:22 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:22.532397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00dad-b2fd-4dc2-840e-cfd3fe8d72e9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.543433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00dad-b2fd-4dc2-840e-cfd3fe8d72e9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.600453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.600977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.601253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.601523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.602168 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.604102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.628568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1300/2612] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:22 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:49:22.632915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.644324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998ff90-36a3-4485-9d2b-f3b3eadb7bb8 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.649685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.650273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.650550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.650820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.651495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.658630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.677062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1313/2613] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:22.682009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.699035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382fda5-bd1c-4e43-9e14-b5d799684351 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.706952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.707979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.708362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.708721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] PATH_INFO: `/v3/projects/7b576be6c5ea4bbc9887f62db292dca3/users/8a9800493f2c431b9f44ff56df56ecae/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.726313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b576be6c5ea4bbc9887f62db292dca3, user_id=8a9800493f2c431b9f44ff56df56ecae, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.729002 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.744452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1301/2614] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:22 2022] PUT /identity/v3/projects/7b576be6c5ea4bbc9887f62db292dca3/users/8a9800493f2c431b9f44ff56df56ecae/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:22.749075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.759811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bf03d-8349-4157-91c5-ef3ecd50973c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.819887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92bd49-0128-4e98-b7c3-9aeda1129b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.820488 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92bd49-0128-4e98-b7c3-9aeda1129b90 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.820763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92bd49-0128-4e98-b7c3-9aeda1129b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.821031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92bd49-0128-4e98-b7c3-9aeda1129b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.821666 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92bd49-0128-4e98-b7c3-9aeda1129b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.823295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92bd49-0128-4e98-b7c3-9aeda1129b90 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.833386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1314/2615] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:22.838076 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:22.850200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a500f29-f56e-4340-91e4-93853733c3a8 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:22.856424 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:22.857017 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.857280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.857553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] PATH_INFO: `/v3/projects/7b576be6c5ea4bbc9887f62db292dca3/users/8a9800493f2c431b9f44ff56df56ecae/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.859826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b576be6c5ea4bbc9887f62db292dca3, user_id=8a9800493f2c431b9f44ff56df56ecae, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:22.862550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:22.873707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1302/2616] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:22 2022] PUT /identity/v3/projects/7b576be6c5ea4bbc9887f62db292dca3/users/8a9800493f2c431b9f44ff56df56ecae/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:22.885050 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc947e67-dbec-458e-be5d-482b4dc76713 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:22.885348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc947e67-dbec-458e-be5d-482b4dc76713 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:22.885635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc947e67-dbec-458e-be5d-482b4dc76713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:22.918205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bc947e67-dbec-458e-be5d-482b4dc76713 None None] MFA Rules not processed for user `8a9800493f2c431b9f44ff56df56ecae`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:22.919452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bc947e67-dbec-458e-be5d-482b4dc76713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:22.996073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:22.999995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1315/2617] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:23.016587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:23.018582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:23.041079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:23.077928 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa33d6-830c-49e6-b2fc-c14d2a7ff8bf admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:23.082097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fd073-e89d-41af-8e31-4ee7a1ce24c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:23.082696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fd073-e89d-41af-8e31-4ee7a1ce24c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:23.082972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fd073-e89d-41af-8e31-4ee7a1ce24c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:23.083238 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fd073-e89d-41af-8e31-4ee7a1ce24c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:23.093314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fd073-e89d-41af-8e31-4ee7a1ce24c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:23.094759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fd073-e89d-41af-8e31-4ee7a1ce24c3 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:23.104208 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1303/2618] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:23.529805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc947e67-dbec-458e-be5d-482b4dc76713 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:23.595861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc947e67-dbec-458e-be5d-482b4dc76713 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:23.600779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:23.601415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:23.601695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:23.601967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] PATH_INFO: `/v3/users/7c490d7b5d334f5fa16d331ed452b944` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:23.603442 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c490d7b5d334f5fa16d331ed452b944)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:23.606053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:23.655019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c490d7b5d334f5fa16d331ed452b944'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:23.655710 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c490d7b5d334f5fa16d331ed452b944'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:23.662626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c490d7b5d334f5fa16d331ed452b944'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:23.663264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c490d7b5d334f5fa16d331ed452b944'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:23.674318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c490d7b5d334f5fa16d331ed452b944'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:23.693086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1316/2619] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:23 2022] DELETE /identity/v3/users/7c490d7b5d334f5fa16d331ed452b944 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:23.697923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:23.698436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:23.698535 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:23.718622 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:23.719874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:23.780239 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1304/2620] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:23.792822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:23.794277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:23.861399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c8926-d79f-4b8f-b325-2af2677ae9d6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:23.865461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:23.866069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:23.866329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:23.866624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:23.876321 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:23.877812 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:23.888032 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1317/2621] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:24.170492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:24.172252 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:24.185172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbb0a3-1518-4334-a1f2-cdc7405bbdf2 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:24.190730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:24.191454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:24.191835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:24.192165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] PATH_INFO: `/v3/projects/4370396362ea4362aa2fbedf8ec59ab3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:24.193454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4370396362ea4362aa2fbedf8ec59ab3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:24.195447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:24.248471 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4370396362ea4362aa2fbedf8ec59ab3', 'request_id': 'req-5698f5a6-2543-40ce-ae1c-c6b19cca056e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.254026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4370396362ea4362aa2fbedf8ec59ab3', 'request_id': 'req-5698f5a6-2543-40ce-ae1c-c6b19cca056e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.260332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4370396362ea4362aa2fbedf8ec59ab3', 'request_id': 'req-5698f5a6-2543-40ce-ae1c-c6b19cca056e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.276957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] The token cache is being invalidate because project 4370396362ea4362aa2fbedf8ec59ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:24.277262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.277949 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.279520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1305/2622] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:24 2022] DELETE /identity/v3/projects/4370396362ea4362aa2fbedf8ec59ab3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:24.283715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:24.293454 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b696-9273-46c5-bad2-e657c6062d69 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:24.311538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:24.356862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:24.357500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:24.357774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:24.358039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] PATH_INFO: `/v3/users/8a9800493f2c431b9f44ff56df56ecae` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:24.359484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9800493f2c431b9f44ff56df56ecae)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:24.361479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:24.410897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9800493f2c431b9f44ff56df56ecae'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.411552 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9800493f2c431b9f44ff56df56ecae'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.418808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9800493f2c431b9f44ff56df56ecae'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.419497 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9800493f2c431b9f44ff56df56ecae'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.428280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-ef87dfbe-c40b-485b-bd35-871797619a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9800493f2c431b9f44ff56df56ecae'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.446583 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1318/2623] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:24 2022] DELETE /identity/v3/users/8a9800493f2c431b9f44ff56df56ecae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:24.730825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:24.732525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:24.749789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698f5a6-2543-40ce-ae1c-c6b19cca056e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:24.794475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:24.809242 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:24.809882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:24.810185 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:24.810452 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] PATH_INFO: `/v3/projects/7b576be6c5ea4bbc9887f62db292dca3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:24.811563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b576be6c5ea4bbc9887f62db292dca3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:24.813502 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:24.863364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b576be6c5ea4bbc9887f62db292dca3', 'request_id': 'req-79033e46-cab0-4031-a766-8f9fa49e719a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.869877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b576be6c5ea4bbc9887f62db292dca3', 'request_id': 'req-79033e46-cab0-4031-a766-8f9fa49e719a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.876770 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b576be6c5ea4bbc9887f62db292dca3', 'request_id': 'req-79033e46-cab0-4031-a766-8f9fa49e719a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.893359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] The token cache is being invalidate because project 7b576be6c5ea4bbc9887f62db292dca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:24.893646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.894289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79033e46-cab0-4031-a766-8f9fa49e719a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:24.895817 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1306/2624] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:24 2022] DELETE /identity/v3/projects/7b576be6c5ea4bbc9887f62db292dca3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:24.906036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f2ed8-91fa-45bd-875c-566f5bf57a6d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:24.906483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f2ed8-91fa-45bd-875c-566f5bf57a6d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:24.906590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f2ed8-91fa-45bd-875c-566f5bf57a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:24.927837 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-6c6f2ed8-91fa-45bd-875c-566f5bf57a6d None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:24.929053 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6c6f2ed8-91fa-45bd-875c-566f5bf57a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:24.994359 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1319/2625] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:25.002534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba882-baa4-4dd2-bc33-9869b5ef497e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.002789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba882-baa4-4dd2-bc33-9869b5ef497e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.003059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba882-baa4-4dd2-bc33-9869b5ef497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.022825 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-cddba882-baa4-4dd2-bc33-9869b5ef497e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:25.024001 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-cddba882-baa4-4dd2-bc33-9869b5ef497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:25.032779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1307/2626] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:25.037162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f2ed8-91fa-45bd-875c-566f5bf57a6d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.047457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f2ed8-91fa-45bd-875c-566f5bf57a6d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.052878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.053506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.053786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.054059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.054708 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:49:25.055010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.056702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.069648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1320/2627] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:49:25.074169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddba882-baa4-4dd2-bc33-9869b5ef497e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.087104 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddba882-baa4-4dd2-bc33-9869b5ef497e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.095961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.096929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.097210 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.097571 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.098214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.101189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.137784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1308/2628] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:25 2022] POST /identity/v3/projects => generated 393 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:25.142171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.152460 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0461d-e2c1-4719-8f3b-69a677d59bdd admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.179226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:25.211352 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.211966 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.212244 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.212512 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.213131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.215701 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.238501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1321/2629] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:25 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:25.242796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.253492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036e716-dea2-4585-b721-c137885c2960 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.259588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.260209 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.260517 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.260755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.261560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.263308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.274136 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1309/2630] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:25.278166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.288323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acb9c7-4368-4f7c-89c2-8ea5388e0715 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.293289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d89672-3794-434d-9992-640adc80c8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.293882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d89672-3794-434d-9992-640adc80c8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.294161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d89672-3794-434d-9992-640adc80c8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.294434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d89672-3794-434d-9992-640adc80c8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.295085 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d89672-3794-434d-9992-640adc80c8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.296750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d89672-3794-434d-9992-640adc80c8a4 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.307444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1322/2631] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:25.311894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.322194 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa50b63-2659-4a44-8a9e-c8e64645aa15 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.327890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.328544 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.328814 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.329077 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] PATH_INFO: `/v3/projects/c7ad1816c2414f5f804ecf3cd0170d91/users/4ac64c85d10142f59a847a03cc5d2b24/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.346100 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ad1816c2414f5f804ecf3cd0170d91, user_id=4ac64c85d10142f59a847a03cc5d2b24, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.350030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.372635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1310/2632] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:25 2022] PUT /identity/v3/projects/c7ad1816c2414f5f804ecf3cd0170d91/users/4ac64c85d10142f59a847a03cc5d2b24/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:25.384726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0734b2-1fb8-4422-8713-f4e6d4dbe192 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.385014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0734b2-1fb8-4422-8713-f4e6d4dbe192 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.385282 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0734b2-1fb8-4422-8713-f4e6d4dbe192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.413928 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-7c0734b2-1fb8-4422-8713-f4e6d4dbe192 None None] MFA Rules not processed for user `4ac64c85d10142f59a847a03cc5d2b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:25.415135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-7c0734b2-1fb8-4422-8713-f4e6d4dbe192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:25.483646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1323/2633] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:25.488200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.552581 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811f8f8-d3bc-4fc7-9e03-df6e753f7985 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.557908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.558512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.558794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.559064 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.559729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.561633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.583685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:25.593216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1311/2634] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:25 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:25.597518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0734b2-1fb8-4422-8713-f4e6d4dbe192 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.608476 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0734b2-1fb8-4422-8713-f4e6d4dbe192 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.667622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.668254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.668531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.668798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.669416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.671333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.702165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1324/2635] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:25 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:25.705653 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.721035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b2e7-8873-4782-8cc9-4f6353901e56 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.727173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.727807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.728088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.728365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.729013 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.730646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.742195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1312/2636] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:25.747651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.757903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fc481-c1f7-4d7e-a7f8-6aa53c70a98a admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.763308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.764052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.764328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.764598 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] PATH_INFO: `/v3/projects/d1331616c8f6405d8652f01ec2fa7561/users/92e3b31e4a584626805cc977fae8f8a8/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.779931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1331616c8f6405d8652f01ec2fa7561, user_id=92e3b31e4a584626805cc977fae8f8a8, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.782563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.797273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1325/2637] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:25 2022] PUT /identity/v3/projects/d1331616c8f6405d8652f01ec2fa7561/users/92e3b31e4a584626805cc977fae8f8a8/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:25.801944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.814914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061286-92f7-4050-aa38-d31e80a4511b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.875966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2f18b-478e-4942-ab90-8365d70443d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.876587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2f18b-478e-4942-ab90-8365d70443d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.876865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2f18b-478e-4942-ab90-8365d70443d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.877137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2f18b-478e-4942-ab90-8365d70443d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.877787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2f18b-478e-4942-ab90-8365d70443d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.879388 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2f18b-478e-4942-ab90-8365d70443d7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.889405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1313/2638] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:25.893753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:25.900205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:25.906657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521335b-1925-4cbb-89d1-b29d156f23e5 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:25.912043 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:25.912722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.912996 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.913259 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] PATH_INFO: `/v3/projects/d1331616c8f6405d8652f01ec2fa7561/users/92e3b31e4a584626805cc977fae8f8a8/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.915237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1331616c8f6405d8652f01ec2fa7561, user_id=92e3b31e4a584626805cc977fae8f8a8, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:25.917940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:25.928363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1326/2639] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:25 2022] PUT /identity/v3/projects/d1331616c8f6405d8652f01ec2fa7561/users/92e3b31e4a584626805cc977fae8f8a8/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:25.939197 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:25.939491 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:25.939795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:25.970617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] MFA Rules not processed for user `92e3b31e4a584626805cc977fae8f8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:25.971857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:26.049068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1314/2640] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:26.064489 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:26.067644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:26.130614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2df09-6179-4c66-ad34-5418c5313258 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:26.134660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:26.135265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:26.135542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:26.135917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:26.145055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:26.146503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:26.156075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1327/2641] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:26 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:26.204377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:26.206880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:26.222405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5d4d3-a758-422d-8a01-f92eec962348 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:26.226842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:26.227455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:26.227775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:26.228050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:26.292055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:26.294204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:26.304516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1315/2642] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:27.049842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:27.051737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:27.113184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f9c91-10e9-4c3b-95d6-7deee31cfd10 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:27.116796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917b6ca-e3bd-49cc-acf3-6d198d83e14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:27.117414 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917b6ca-e3bd-49cc-acf3-6d198d83e14f service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:27.117693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917b6ca-e3bd-49cc-acf3-6d198d83e14f service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:27.117949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917b6ca-e3bd-49cc-acf3-6d198d83e14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:27.119902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d917b6ca-e3bd-49cc-acf3-6d198d83e14f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:27.131452 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917b6ca-e3bd-49cc-acf3-6d198d83e14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:27.133028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917b6ca-e3bd-49cc-acf3-6d198d83e14f service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:27.141736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1328/2643] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:29.686442 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:29.697073 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:29.761009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b7e6f-75da-44cc-8da1-a008097857d4 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:29.766943 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:29.767597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:29.767879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:29.768153 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] PATH_INFO: `/v3/users/4ac64c85d10142f59a847a03cc5d2b24` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:29.769312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac64c85d10142f59a847a03cc5d2b24)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:29.771222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:29.820861 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac64c85d10142f59a847a03cc5d2b24'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:29.821515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac64c85d10142f59a847a03cc5d2b24'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:29.822103 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac64c85d10142f59a847a03cc5d2b24'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:29.831563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac64c85d10142f59a847a03cc5d2b24'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:29.839708 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac64c85d10142f59a847a03cc5d2b24'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:29.856819 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1316/2644] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:29 2022] DELETE /identity/v3/users/4ac64c85d10142f59a847a03cc5d2b24 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:29.861441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:29.861921 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:29.862015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:29.881995 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:29.883200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:29.948307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1329/2645] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:29.958362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:29.959862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:30.018766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aff5c9-f166-4bb5-b53f-73519e39f8b8 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:30.022860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:30.023475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:30.023797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:30.024046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:30.033354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:30.034781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:30.044108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1317/2646] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:30.334631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:30.336209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:30.349263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbffcc-f9b2-4e5e-b3a8-44f42bd48bd9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:30.354814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:30.355466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:30.355890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:30.356205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] PATH_INFO: `/v3/projects/c7ad1816c2414f5f804ecf3cd0170d91` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:30.357569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ad1816c2414f5f804ecf3cd0170d91)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:30.359628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:30.395079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:30.409286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ad1816c2414f5f804ecf3cd0170d91', 'request_id': 'req-3f55d144-f05d-4c08-ba8e-fd63c900ca15'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.414722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ad1816c2414f5f804ecf3cd0170d91', 'request_id': 'req-3f55d144-f05d-4c08-ba8e-fd63c900ca15'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.421532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ad1816c2414f5f804ecf3cd0170d91', 'request_id': 'req-3f55d144-f05d-4c08-ba8e-fd63c900ca15'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.435951 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] The token cache is being invalidate because project c7ad1816c2414f5f804ecf3cd0170d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:30.436241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.437090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.439418 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1330/2647] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:30 2022] DELETE /identity/v3/projects/c7ad1816c2414f5f804ecf3cd0170d91 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:30.445739 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:30.457932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8782372-a172-49e2-89ae-23b159680cb8 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:30.487040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:30.520148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:30.520864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:30.521139 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:30.521406 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] PATH_INFO: `/v3/users/92e3b31e4a584626805cc977fae8f8a8` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:30.522499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e3b31e4a584626805cc977fae8f8a8)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:30.524518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:30.577220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e3b31e4a584626805cc977fae8f8a8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.577870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e3b31e4a584626805cc977fae8f8a8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.578453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e3b31e4a584626805cc977fae8f8a8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.587159 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e3b31e4a584626805cc977fae8f8a8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.595284 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-62a5ef2d-5649-4ed2-812f-26ee75ff0091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e3b31e4a584626805cc977fae8f8a8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:30.613759 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1318/2648] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:30 2022] DELETE /identity/v3/users/92e3b31e4a584626805cc977fae8f8a8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:30.900679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:30.902312 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:30.919116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55d144-f05d-4c08-ba8e-fd63c900ca15 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:30.980264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:30.980914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:30.981165 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:30.981446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] PATH_INFO: `/v3/projects/d1331616c8f6405d8652f01ec2fa7561` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:30.982637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1331616c8f6405d8652f01ec2fa7561)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:30.984610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.032213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1331616c8f6405d8652f01ec2fa7561', 'request_id': 'req-7295327c-4a35-4d9b-a2d6-c24ca37a4524'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:31.040423 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1331616c8f6405d8652f01ec2fa7561', 'request_id': 'req-7295327c-4a35-4d9b-a2d6-c24ca37a4524'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:31.047815 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1331616c8f6405d8652f01ec2fa7561', 'request_id': 'req-7295327c-4a35-4d9b-a2d6-c24ca37a4524'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:31.064071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] The token cache is being invalidate because project d1331616c8f6405d8652f01ec2fa7561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:31.064430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:31.065094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7295327c-4a35-4d9b-a2d6-c24ca37a4524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:31.066730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1331/2649] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:30 2022] DELETE /identity/v3/projects/d1331616c8f6405d8652f01ec2fa7561 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:49:31.072713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d884-ba63-49ea-933c-c553ee4d732b None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.073183 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d884-ba63-49ea-933c-c553ee4d732b None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.073315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d884-ba63-49ea-933c-c553ee4d732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.094788 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-aeb8d884-ba63-49ea-933c-c553ee4d732b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:31.096026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aeb8d884-ba63-49ea-933c-c553ee4d732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:31.164021 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1319/2650] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:31.173480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f6b95-9603-4cde-a47c-e6806dac7917 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.173764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f6b95-9603-4cde-a47c-e6806dac7917 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.174033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f6b95-9603-4cde-a47c-e6806dac7917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.194715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-756f6b95-9603-4cde-a47c-e6806dac7917 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:31.196008 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-756f6b95-9603-4cde-a47c-e6806dac7917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:31.204985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1332/2651] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:31.209529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8d884-ba63-49ea-933c-c553ee4d732b None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.220055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8d884-ba63-49ea-933c-c553ee4d732b None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.228266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.229141 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.229504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.229833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.230679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:49:31.231121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.233052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.245670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1320/2652] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 05:49:31.250306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f6b95-9603-4cde-a47c-e6806dac7917 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.260138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f6b95-9603-4cde-a47c-e6806dac7917 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.266141 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.266747 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.267062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.267334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.268019 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.271164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.305587 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1333/2653] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:31 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:31.312612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.328652 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029352-20df-4411-bbbf-44b6c38d2cdc admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.362725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:31.392068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.392678 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.392959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.393231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.393860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.395866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.419343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1321/2654] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:31 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:31.423669 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.434649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6e37-996b-4156-a8b1-30a9abf6537b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.440051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.440660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.440939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.441211 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.441860 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.443487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.454508 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1334/2655] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:31.458834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.470020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a155756-53da-44c8-914b-d02f27f0915b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.475966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d2b5d-cc68-469d-ab02-98bd13028a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.476578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2b5d-cc68-469d-ab02-98bd13028a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.476853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2b5d-cc68-469d-ab02-98bd13028a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.477123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2b5d-cc68-469d-ab02-98bd13028a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.477771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d2b5d-cc68-469d-ab02-98bd13028a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.479382 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d2b5d-cc68-469d-ab02-98bd13028a9d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.489674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1322/2656] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:31.493750 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.503991 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84c5d9-a665-4a89-b939-1c575c724252 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.508697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.509342 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.509635 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.509908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] PATH_INFO: `/v3/projects/27b9575f268647b69d80b5deea6bd637/users/3633056875c846bc9dd27ac0b166fb75/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.512973 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:31.527637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b9575f268647b69d80b5deea6bd637, user_id=3633056875c846bc9dd27ac0b166fb75, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.530481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.544129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1335/2657] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:31 2022] PUT /identity/v3/projects/27b9575f268647b69d80b5deea6bd637/users/3633056875c846bc9dd27ac0b166fb75/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:31.554758 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd42a-9f6f-4485-b2bf-806672f26712 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.555049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd42a-9f6f-4485-b2bf-806672f26712 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.555325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd42a-9f6f-4485-b2bf-806672f26712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.585851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-849fd42a-9f6f-4485-b2bf-806672f26712 None None] MFA Rules not processed for user `3633056875c846bc9dd27ac0b166fb75`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:31.586976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-849fd42a-9f6f-4485-b2bf-806672f26712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:31.657020 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1323/2658] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:31.661719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.727905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24b8c7-b596-40f1-bdac-299c0a985273 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.733264 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.733927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.734202 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.734475 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.735101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.737034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.765905 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1336/2659] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:31 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:31.770009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fd42a-9f6f-4485-b2bf-806672f26712 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.781514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fd42a-9f6f-4485-b2bf-806672f26712 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.847049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.847601 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.847882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.848160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.848789 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.850712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.875524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1324/2660] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:31 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:49:31.880421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.895340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd5607-eda3-4cff-8f3f-42ec9db65dd2 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.901435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.902203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.902515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.902842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.903606 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.905298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.917305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1337/2661] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:31.921832 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.932333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390dc85-b78c-4e3e-9cd8-1b5af77b63a5 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:31.938051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:31.938712 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:31.939107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:31.939243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] PATH_INFO: `/v3/projects/91f2c50e9afe4657ac50a4a0d446f9c2/users/532ce22e62b74a7facac317d7aab7d51/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:31.954784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f2c50e9afe4657ac50a4a0d446f9c2, user_id=532ce22e62b74a7facac317d7aab7d51, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:31.957459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:31.973533 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1325/2662] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:31 2022] PUT /identity/v3/projects/91f2c50e9afe4657ac50a4a0d446f9c2/users/532ce22e62b74a7facac317d7aab7d51/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:31.977679 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:31.989099 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf96ab-7d61-4492-bf2d-d2a65230b984 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:32.047983 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6b6e4-dff9-4a51-9351-634b1cad68cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:32.048585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6b6e4-dff9-4a51-9351-634b1cad68cb admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:32.048873 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6b6e4-dff9-4a51-9351-634b1cad68cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:32.049157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6b6e4-dff9-4a51-9351-634b1cad68cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:32.049781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6b6e4-dff9-4a51-9351-634b1cad68cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:32.051386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6b6e4-dff9-4a51-9351-634b1cad68cb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:32.061296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1338/2663] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:32.067360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:32.079150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd27c7-c711-44d3-b96e-36935da77a97 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:32.085214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:32.085914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:32.086222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:32.086492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] PATH_INFO: `/v3/projects/91f2c50e9afe4657ac50a4a0d446f9c2/users/532ce22e62b74a7facac317d7aab7d51/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:32.089164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f2c50e9afe4657ac50a4a0d446f9c2, user_id=532ce22e62b74a7facac317d7aab7d51, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:32.091848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:32.095057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:32.103643 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1326/2664] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:32 2022] PUT /identity/v3/projects/91f2c50e9afe4657ac50a4a0d446f9c2/users/532ce22e62b74a7facac317d7aab7d51/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:32.114425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071d39a-df3b-460d-8569-e2d65ba31b97 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:32.114714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071d39a-df3b-460d-8569-e2d65ba31b97 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:32.114982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071d39a-df3b-460d-8569-e2d65ba31b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:32.152028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-2071d39a-df3b-460d-8569-e2d65ba31b97 None None] MFA Rules not processed for user `532ce22e62b74a7facac317d7aab7d51`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:32.153796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2071d39a-df3b-460d-8569-e2d65ba31b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:32.199933 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:32.227474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1339/2665] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:32.240135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:32.241984 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:32.311996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001ac50-ddfb-4224-87d0-ae86fcd9a3e9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:32.316178 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:32.316777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:32.317055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:32.317318 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:32.327188 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:32.328879 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:32.339235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1327/2666] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:32.440356 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071d39a-df3b-460d-8569-e2d65ba31b97 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:32.452195 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071d39a-df3b-460d-8569-e2d65ba31b97 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:32.456505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf0b18-5230-4c6c-a537-dcba5e4b150f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:32.457114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0b18-5230-4c6c-a537-dcba5e4b150f service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:32.457400 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0b18-5230-4c6c-a537-dcba5e4b150f service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:32.457673 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0b18-5230-4c6c-a537-dcba5e4b150f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:32.516177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf0b18-5230-4c6c-a537-dcba5e4b150f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:32.517633 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf0b18-5230-4c6c-a537-dcba5e4b150f service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:32.526642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1340/2667] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:32.805237 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:32.866537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19c459-9d82-4c3f-9b99-678e4a67e329 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:32.871523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:32.872179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:32.872519 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:32.872689 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] PATH_INFO: `/v3/users/3633056875c846bc9dd27ac0b166fb75` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:32.873864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3633056875c846bc9dd27ac0b166fb75)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:32.875858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:32.922704 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3633056875c846bc9dd27ac0b166fb75'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:32.923356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3633056875c846bc9dd27ac0b166fb75'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:32.923932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3633056875c846bc9dd27ac0b166fb75'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:32.931591 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3633056875c846bc9dd27ac0b166fb75'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:32.938831 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3633056875c846bc9dd27ac0b166fb75'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:32.957646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1328/2668] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:32 2022] DELETE /identity/v3/users/3633056875c846bc9dd27ac0b166fb75 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:32.962308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:32.962790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:32.962915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:32.983565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:32.984931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:33.048545 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1341/2669] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:33.060037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:33.061495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:33.083852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:33.123439 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9038cb-c50b-453d-9edb-ea44467ef2ec admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:33.127764 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:33.128780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:33.129262 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:33.129737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:33.142701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:33.144518 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:33.157016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1329/2670] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:33.442570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:33.444287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:33.458630 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb705e5c-cbb1-4e22-8722-4d00ddd7a8a4 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:33.464473 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:33.465648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:33.466088 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:33.466360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] PATH_INFO: `/v3/projects/27b9575f268647b69d80b5deea6bd637` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:33.467601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b9575f268647b69d80b5deea6bd637)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:33.469971 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:33.519459 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b9575f268647b69d80b5deea6bd637', 'request_id': 'req-cf3881ff-6d3c-49bb-af75-f992d19151b2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.525760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b9575f268647b69d80b5deea6bd637', 'request_id': 'req-cf3881ff-6d3c-49bb-af75-f992d19151b2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.532371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b9575f268647b69d80b5deea6bd637', 'request_id': 'req-cf3881ff-6d3c-49bb-af75-f992d19151b2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.547034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] The token cache is being invalidate because project 27b9575f268647b69d80b5deea6bd637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:33.547314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.547952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.549502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1342/2671] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:33 2022] DELETE /identity/v3/projects/27b9575f268647b69d80b5deea6bd637 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:33.553644 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:33.564177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff7f1b-40d2-4ca8-9fa6-4b58834afbd0 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:33.626959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:33.627606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:33.627918 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:33.628174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] PATH_INFO: `/v3/users/532ce22e62b74a7facac317d7aab7d51` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:33.629361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=532ce22e62b74a7facac317d7aab7d51)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:33.631302 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:33.680445 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532ce22e62b74a7facac317d7aab7d51'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.681078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532ce22e62b74a7facac317d7aab7d51'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.681663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532ce22e62b74a7facac317d7aab7d51'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.689615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532ce22e62b74a7facac317d7aab7d51'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.696290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-5e3d6621-8b7b-486b-a6a9-0bf8638fab3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532ce22e62b74a7facac317d7aab7d51'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:33.715084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1330/2672] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:33 2022] DELETE /identity/v3/users/532ce22e62b74a7facac317d7aab7d51 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:33.993250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:33.994796 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:34.003765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:34.010069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3881ff-6d3c-49bb-af75-f992d19151b2 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.074313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.074956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.075233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.075501 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] PATH_INFO: `/v3/projects/91f2c50e9afe4657ac50a4a0d446f9c2` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.076698 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f2c50e9afe4657ac50a4a0d446f9c2)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.079435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.127106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f2c50e9afe4657ac50a4a0d446f9c2', 'request_id': 'req-a7460681-0c03-4a94-bd4c-c10640497ee6'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:34.132523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f2c50e9afe4657ac50a4a0d446f9c2', 'request_id': 'req-a7460681-0c03-4a94-bd4c-c10640497ee6'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:34.138542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f2c50e9afe4657ac50a4a0d446f9c2', 'request_id': 'req-a7460681-0c03-4a94-bd4c-c10640497ee6'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:34.152608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] The token cache is being invalidate because project 91f2c50e9afe4657ac50a4a0d446f9c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:34.152893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:34.153502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-a7460681-0c03-4a94-bd4c-c10640497ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:34.155007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1343/2673] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:33 2022] DELETE /identity/v3/projects/91f2c50e9afe4657ac50a4a0d446f9c2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:34.160698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17809fc-9bdd-4049-b64e-d66138bdb2d1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.161175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17809fc-9bdd-4049-b64e-d66138bdb2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.161273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17809fc-9bdd-4049-b64e-d66138bdb2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.181470 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e17809fc-9bdd-4049-b64e-d66138bdb2d1 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:34.182674 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e17809fc-9bdd-4049-b64e-d66138bdb2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:34.216932 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:34.245598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1331/2674] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:49:34.254390 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce3f0-5a0e-43c0-9073-6633800f047b None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.254677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce3f0-5a0e-43c0-9073-6633800f047b None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.254952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce3f0-5a0e-43c0-9073-6633800f047b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.276802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f5cce3f0-5a0e-43c0-9073-6633800f047b None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:34.278144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f5cce3f0-5a0e-43c0-9073-6633800f047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:34.288541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1344/2675] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:34.292881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17809fc-9bdd-4049-b64e-d66138bdb2d1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.303410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17809fc-9bdd-4049-b64e-d66138bdb2d1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.309081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.309691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.309967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.310236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.310866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:49:34.311172 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.313359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.327870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1332/2676] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:49:34.332362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce3f0-5a0e-43c0-9073-6633800f047b None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.344184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce3f0-5a0e-43c0-9073-6633800f047b None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.349206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.349836 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.350111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.350381 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.351005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.352949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.385935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1345/2677] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:34 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:34.392040 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.406848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc10b6f-e9c5-4d63-96f3-cf2e3b178f9d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.469245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.469854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.470130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.470397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.471018 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.472967 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.495396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1333/2678] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:34 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:34.500340 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.510791 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f489c4-0903-486e-ab91-67fb6c8a7444 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.516396 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.516999 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.517269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.517531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.518170 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.519817 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.530382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1346/2679] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:34.534516 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.545146 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d993a-22f7-4f89-9df3-8c55f73c909b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.550173 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc6f0f-7a39-4b2c-bde0-1d56202b3b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.550774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc6f0f-7a39-4b2c-bde0-1d56202b3b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.551049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc6f0f-7a39-4b2c-bde0-1d56202b3b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.551320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc6f0f-7a39-4b2c-bde0-1d56202b3b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.552005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc6f0f-7a39-4b2c-bde0-1d56202b3b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.553592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc6f0f-7a39-4b2c-bde0-1d56202b3b5c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.564866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1334/2680] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:34.569115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.579967 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb291adb-07a9-4235-8537-cdd73eb26780 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.586166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.586807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.587072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.587364 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] PATH_INFO: `/v3/projects/ff0ecf5db00b4187aef5afaad2d9d145/users/969e089963b34639a6b39213cfe1a3a0/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.603810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0ecf5db00b4187aef5afaad2d9d145, user_id=969e089963b34639a6b39213cfe1a3a0, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.606490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.621471 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1347/2681] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:34 2022] PUT /identity/v3/projects/ff0ecf5db00b4187aef5afaad2d9d145/users/969e089963b34639a6b39213cfe1a3a0/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:34.634007 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f4001-4387-498d-862a-17fd30f79c92 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.634289 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f4001-4387-498d-862a-17fd30f79c92 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.634556 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f4001-4387-498d-862a-17fd30f79c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.664476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-7c6f4001-4387-498d-862a-17fd30f79c92 None None] MFA Rules not processed for user `969e089963b34639a6b39213cfe1a3a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:34.665679 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7c6f4001-4387-498d-862a-17fd30f79c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:34.736157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1335/2682] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:34.740753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.771972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:34.805946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ef0c8-5800-40de-9850-b372eb784e75 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.811199 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.811851 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.812334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.812691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.813333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.815263 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.844691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1348/2683] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:34 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:34.850445 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f4001-4387-498d-862a-17fd30f79c92 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.862301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f4001-4387-498d-862a-17fd30f79c92 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.897294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:34.927265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.927921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.928200 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.928468 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.929088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.931035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:34.955870 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1336/2684] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:34 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:34.961955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:34.977955 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a24a-0ed3-4517-8f10-24ad72270390 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:34.985325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:34.986393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:34.986741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:34.987595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:34.988716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:34.991194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.003487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1349/2685] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:35.007993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.018155 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778cdcf-7aca-41da-807f-8eca2df33c4f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.022813 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec70fc-e12c-44a6-b70a-0102adb01e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.023365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec70fc-e12c-44a6-b70a-0102adb01e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.023672 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec70fc-e12c-44a6-b70a-0102adb01e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.023944 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec70fc-e12c-44a6-b70a-0102adb01e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.024584 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec70fc-e12c-44a6-b70a-0102adb01e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.026179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec70fc-e12c-44a6-b70a-0102adb01e49 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.038023 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1337/2686] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:49:35.042565 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.052931 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bdcba-1879-4cf7-ac1c-6fd0037050a0 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.057582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.058253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.058514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.058813 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] PATH_INFO: `/v3/projects/23c521b5fc8c49e2a00a5cf9418b8999/users/0aa378bb494349ea97338bf7055add70/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.074271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c521b5fc8c49e2a00a5cf9418b8999, user_id=0aa378bb494349ea97338bf7055add70, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.078809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.095589 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1350/2687] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:35 2022] PUT /identity/v3/projects/23c521b5fc8c49e2a00a5cf9418b8999/users/0aa378bb494349ea97338bf7055add70/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:49:35.168345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b902-d98e-4ba8-9eb4-d6fd7a3232da None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.168960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b902-d98e-4ba8-9eb4-d6fd7a3232da None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.168960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b902-d98e-4ba8-9eb4-d6fd7a3232da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.197489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-9600b902-d98e-4ba8-9eb4-d6fd7a3232da None None] MFA Rules not processed for user `0aa378bb494349ea97338bf7055add70`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:35.198736 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9600b902-d98e-4ba8-9eb4-d6fd7a3232da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:35.264226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1338/2688] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:35.269449 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.342670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863d892-45ac-46b5-a42e-8c81cb3bddca admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.349116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.349759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.350039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.350314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.350969 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.352945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.384120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1351/2689] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:35 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:35.388368 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b902-d98e-4ba8-9eb4-d6fd7a3232da None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.399895 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b902-d98e-4ba8-9eb4-d6fd7a3232da None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.464271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.464880 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.465154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.465421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.466043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.467981 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.491298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1339/2690] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:35 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:35.495694 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.506997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a6c24-85f0-438a-9c19-f95101620c0f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.512664 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.513293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.513574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.513840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.514490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.516296 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.527899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1352/2691] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:35.532154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.542840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fb4e0-24e0-4578-a836-c3af03a8f96e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.548867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.549513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.549800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.550052 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] PATH_INFO: `/v3/projects/d853c389a2e64699a9136ac7bd810deb/users/273e92cb8bfe460ea663de79c759d6d9/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.573260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d853c389a2e64699a9136ac7bd810deb, user_id=273e92cb8bfe460ea663de79c759d6d9, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.575936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.594496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1340/2692] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:35 2022] PUT /identity/v3/projects/d853c389a2e64699a9136ac7bd810deb/users/273e92cb8bfe460ea663de79c759d6d9/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:35.599224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.613794 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c29d3-6fd0-4e7d-9298-eb6b24eb3683 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.668075 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:35.680048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f228a-b878-4b9b-82a0-6eb207645d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.681055 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f228a-b878-4b9b-82a0-6eb207645d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.681391 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f228a-b878-4b9b-82a0-6eb207645d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.681661 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f228a-b878-4b9b-82a0-6eb207645d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.682562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f228a-b878-4b9b-82a0-6eb207645d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.685171 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f228a-b878-4b9b-82a0-6eb207645d68 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.699215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1353/2693] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:35.703908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.715055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ecdf5-ca37-4fef-bb0c-86cc7d4532c0 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.720256 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.721012 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.721362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.721707 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] PATH_INFO: `/v3/projects/d853c389a2e64699a9136ac7bd810deb/users/273e92cb8bfe460ea663de79c759d6d9/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.723864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d853c389a2e64699a9136ac7bd810deb, user_id=273e92cb8bfe460ea663de79c759d6d9, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:35.726569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:35.730090 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:35.739340 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1341/2694] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:35 2022] PUT /identity/v3/projects/d853c389a2e64699a9136ac7bd810deb/users/273e92cb8bfe460ea663de79c759d6d9/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:49:35.749574 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.750037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.750498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:35.780175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] MFA Rules not processed for user `273e92cb8bfe460ea663de79c759d6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:35.781377 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:35.855908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1354/2695] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:35.871112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:35.873539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:35.935828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8312-37c7-4d5a-8723-2ae1dd3b8e67 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:35.940365 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881aced-8caf-4c81-a799-874a92ca571e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:35.940972 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881aced-8caf-4c81-a799-874a92ca571e service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:35.941243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881aced-8caf-4c81-a799-874a92ca571e service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:35.941507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881aced-8caf-4c81-a799-874a92ca571e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:36.005016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881aced-8caf-4c81-a799-874a92ca571e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:36.006454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881aced-8caf-4c81-a799-874a92ca571e service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:36.016068 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1342/2696] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:37.543490 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:37.544940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:37.557946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c0579-deb7-4446-97fe-9dd4cfb45ff6 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:37.561822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:37.562561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:37.563204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:37.563204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:37.626365 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:37.627866 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:37.637746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1355/2697] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:37.880486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:37.881777 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:37.882507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:37.909543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] MFA Rules not processed for user `ed76a574d2fc4bd8a1f1176d0fef86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:37.910766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:37.974713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1343/2698] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:49:38.200169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:38.209523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228b9ca-fce2-4bac-b5b0-0ef5984b59de service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:38.213834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:38.214463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:38.214718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:38.214985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:38.224908 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:38.226370 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:38.236225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1356/2699] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:38.301957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:38.303457 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:38.369937 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a8d90-bf6c-45e8-8f66-c94f0ee6ffdc None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:38.373512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:38.374122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:38.374398 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:38.374826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:38.376588 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:38.386224 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:38.387703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:38.391221 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=100 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:38.397748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1344/2700] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:38.764255 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:38.765986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:38.837048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:38.843786 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279275-01f9-4102-b400-20a6eef9d6a1 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:38.848762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ddfe1-0ac4-48c5-9d78-d6ab9f2c6c5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:38.849401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ddfe1-0ac4-48c5-9d78-d6ab9f2c6c5b service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:38.849677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ddfe1-0ac4-48c5-9d78-d6ab9f2c6c5b service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:38.849945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ddfe1-0ac4-48c5-9d78-d6ab9f2c6c5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:38.851735 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d00ddfe1-0ac4-48c5-9d78-d6ab9f2c6c5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:38.862965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ddfe1-0ac4-48c5-9d78-d6ab9f2c6c5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:38.864657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ddfe1-0ac4-48c5-9d78-d6ab9f2c6c5b service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:38.874275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1357/2701] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:42.183617 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:42.249313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088d266-f573-4a68-be3d-533767de8e80 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:42.254074 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:42.254765 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:42.255037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:42.255299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] PATH_INFO: `/v3/users/969e089963b34639a6b39213cfe1a3a0` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:42.256525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=969e089963b34639a6b39213cfe1a3a0)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:42.258443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:42.307902 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '969e089963b34639a6b39213cfe1a3a0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.308568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '969e089963b34639a6b39213cfe1a3a0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.309216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '969e089963b34639a6b39213cfe1a3a0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.317184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '969e089963b34639a6b39213cfe1a3a0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.324203 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '969e089963b34639a6b39213cfe1a3a0'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.341977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1345/2702] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:42 2022] DELETE /identity/v3/users/969e089963b34639a6b39213cfe1a3a0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:42.346830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:42.347363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:42.347363 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:42.366612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:42.367845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:42.430143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1358/2703] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:42.440198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:42.442091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:42.500376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541bc97-57ed-4b27-b26a-28ee12a59d4b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:42.506822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:42.507774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:42.508048 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:42.508367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:42.521853 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:42.523279 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:42.532722 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1346/2704] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:42.832504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:42.833956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:42.846855 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e16db-3ddb-4575-84af-db1c04eea5ed None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:42.852355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:42.853007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:42.853288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:42.853563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] PATH_INFO: `/v3/projects/ff0ecf5db00b4187aef5afaad2d9d145` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:42.854723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0ecf5db00b4187aef5afaad2d9d145)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:42.856702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:42.911568 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0ecf5db00b4187aef5afaad2d9d145', 'request_id': 'req-8b496639-74f6-4b34-a13e-e56d3a377351'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.919362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0ecf5db00b4187aef5afaad2d9d145', 'request_id': 'req-8b496639-74f6-4b34-a13e-e56d3a377351'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.926287 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0ecf5db00b4187aef5afaad2d9d145', 'request_id': 'req-8b496639-74f6-4b34-a13e-e56d3a377351'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.941250 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] The token cache is being invalidate because project ff0ecf5db00b4187aef5afaad2d9d145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:42.941456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.942076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:42.943634 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1359/2705] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:42 2022] DELETE /identity/v3/projects/ff0ecf5db00b4187aef5afaad2d9d145 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:42.947791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:42.958005 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86fefb-bbf4-486a-82b5-81638a5d8b8e service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:42.975328 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:43.024164 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:43.024829 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:43.025109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:43.025383 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] PATH_INFO: `/v3/users/0aa378bb494349ea97338bf7055add70` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:43.026547 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa378bb494349ea97338bf7055add70)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:43.028550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:43.077222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa378bb494349ea97338bf7055add70'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.077848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa378bb494349ea97338bf7055add70'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.078444 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa378bb494349ea97338bf7055add70'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.087834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa378bb494349ea97338bf7055add70'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.095099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa378bb494349ea97338bf7055add70'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.112165 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1347/2706] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:42 2022] DELETE /identity/v3/users/0aa378bb494349ea97338bf7055add70 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:43.475960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:43.477672 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:43.491693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b496639-74f6-4b34-a13e-e56d3a377351 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:43.561870 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:43.562519 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:43.562795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:43.563066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] PATH_INFO: `/v3/projects/23c521b5fc8c49e2a00a5cf9418b8999` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:43.564385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c521b5fc8c49e2a00a5cf9418b8999)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:43.566619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:43.582956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:43.618129 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c521b5fc8c49e2a00a5cf9418b8999', 'request_id': 'req-cdad216d-6ec1-42c0-b05f-c8374cb12566'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.623956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c521b5fc8c49e2a00a5cf9418b8999', 'request_id': 'req-cdad216d-6ec1-42c0-b05f-c8374cb12566'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.630781 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c521b5fc8c49e2a00a5cf9418b8999', 'request_id': 'req-cdad216d-6ec1-42c0-b05f-c8374cb12566'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.645224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] The token cache is being invalidate because project 23c521b5fc8c49e2a00a5cf9418b8999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:43.645515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.646168 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.647733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1360/2707] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:43 2022] DELETE /identity/v3/projects/23c521b5fc8c49e2a00a5cf9418b8999 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:43.651796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:43.653474 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:43.666881 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f456b8-4dbb-4108-bd60-686e3f474797 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:43.728525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:43.729171 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:43.729447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:43.729748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] PATH_INFO: `/v3/users/273e92cb8bfe460ea663de79c759d6d9` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:43.731512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273e92cb8bfe460ea663de79c759d6d9)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:43.733480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:43.778747 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:43.786246 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273e92cb8bfe460ea663de79c759d6d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.787108 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273e92cb8bfe460ea663de79c759d6d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.788096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273e92cb8bfe460ea663de79c759d6d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.796942 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273e92cb8bfe460ea663de79c759d6d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.804179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3d4ee735-28d1-47e5-9966-779f44dc0380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273e92cb8bfe460ea663de79c759d6d9'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:43.822015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1348/2708] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:49:43 2022] DELETE /identity/v3/users/273e92cb8bfe460ea663de79c759d6d9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:49:44.112102 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.113620 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:44.129554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad216d-6ec1-42c0-b05f-c8374cb12566 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.190427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.191111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.191385 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.191686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] PATH_INFO: `/v3/projects/d853c389a2e64699a9136ac7bd810deb` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.192917 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d853c389a2e64699a9136ac7bd810deb)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.194829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.244528 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd853c389a2e64699a9136ac7bd810deb', 'request_id': 'req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:44.250807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd853c389a2e64699a9136ac7bd810deb', 'request_id': 'req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:44.257333 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd853c389a2e64699a9136ac7bd810deb', 'request_id': 'req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:44.279399 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] The token cache is being invalidate because project d853c389a2e64699a9136ac7bd810deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:49:44.279656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:44.280697 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-9b12dd3d-cc93-47b6-bd92-5f527e3c881a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:49:44.282997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1361/2709] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:49:44 2022] DELETE /identity/v3/projects/d853c389a2e64699a9136ac7bd810deb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:44.290597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f9288-34fe-4d6f-90c8-637f6efbcbff None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.290980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f9288-34fe-4d6f-90c8-637f6efbcbff None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.291245 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f9288-34fe-4d6f-90c8-637f6efbcbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.319505 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0a5f9288-34fe-4d6f-90c8-637f6efbcbff None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:44.320741 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0a5f9288-34fe-4d6f-90c8-637f6efbcbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:44.383964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1349/2710] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:44.392114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2cd5a-ffc9-45e4-a598-597da9b07b29 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.392401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2cd5a-ffc9-45e4-a598-597da9b07b29 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.392651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2cd5a-ffc9-45e4-a598-597da9b07b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.413539 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bff2cd5a-ffc9-45e4-a598-597da9b07b29 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:44.414734 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bff2cd5a-ffc9-45e4-a598-597da9b07b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:44.422828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1362/2711] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:44.427226 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f9288-34fe-4d6f-90c8-637f6efbcbff None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.437167 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f9288-34fe-4d6f-90c8-637f6efbcbff None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.442057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.442658 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.442936 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.443207 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.443912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:49:44.444215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.445823 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.458815 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1350/2712] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:49:44.463057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2cd5a-ffc9-45e4-a598-597da9b07b29 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.473188 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2cd5a-ffc9-45e4-a598-597da9b07b29 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.478297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.478926 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.479161 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.479428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.480082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.481986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.510838 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1363/2713] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:44 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:44.515127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.526433 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7071130-6a2f-446a-8e19-7f30a8776b1b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.585866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.586482 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.586755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.587022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.587695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.589994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.613326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1351/2714] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:44 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:49:44.617114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.624074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:44.629059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2146f-575b-4003-af11-8a268aa0ba85 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.636416 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.637153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.637562 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.638020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.638745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.640845 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.651822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1364/2715] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:44.655632 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.666184 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc68e85-8ff6-4446-8b04-b90b9d2db8f0 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.670982 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfebc4-fe49-47bc-9b85-741152d96496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.671696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfebc4-fe49-47bc-9b85-741152d96496 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.672035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfebc4-fe49-47bc-9b85-741152d96496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.672367 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfebc4-fe49-47bc-9b85-741152d96496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.673285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfebc4-fe49-47bc-9b85-741152d96496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.675656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfebc4-fe49-47bc-9b85-741152d96496 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.687235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1352/2716] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:44.691546 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.701657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f357f76-5eb3-4936-89cb-d0e01d16b53c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.706887 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.707542 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.707844 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.708115 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] PATH_INFO: `/v3/projects/6d3a9f50ec2d4fff938fcd6a79f5cdc9/users/bb6d35f57b804ba5afa4ab86c46cd7c3/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.723403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3a9f50ec2d4fff938fcd6a79f5cdc9, user_id=bb6d35f57b804ba5afa4ab86c46cd7c3, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.726071 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.743613 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1365/2717] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:44 2022] PUT /identity/v3/projects/6d3a9f50ec2d4fff938fcd6a79f5cdc9/users/bb6d35f57b804ba5afa4ab86c46cd7c3/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:44.748940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.749231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.749504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.769219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:44.770438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:44.792298 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:44.837575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1353/2718] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:44.849028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:44.851965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:44.919961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbec3c-78dc-4faa-869d-60111a396a5b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:44.923832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f8d06-7643-40e8-ab78-42c64a52d525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:44.924458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8d06-7643-40e8-ab78-42c64a52d525 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:44.924738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8d06-7643-40e8-ab78-42c64a52d525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:44.925006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8d06-7643-40e8-ab78-42c64a52d525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:44.934162 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f8d06-7643-40e8-ab78-42c64a52d525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:44.935619 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f8d06-7643-40e8-ab78-42c64a52d525 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:44.945930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1366/2719] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:48.577257 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:48.578816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:48.675169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd2ed4-77d4-46ff-a0d0-0242ae01aebb None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:48.681931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:48.682854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:48.683282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:48.683848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:48.685968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:48.701858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:48.704006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:48.718524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1354/2720] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:49.107835 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e98f2-6778-4c80-9d56-fb2252ee0bc9 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:49.108505 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e98f2-6778-4c80-9d56-fb2252ee0bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:49.108772 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e98f2-6778-4c80-9d56-fb2252ee0bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:49.138890 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f66e98f2-6778-4c80-9d56-fb2252ee0bc9 None None] MFA Rules not processed for user `bb6d35f57b804ba5afa4ab86c46cd7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:49.140069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f66e98f2-6778-4c80-9d56-fb2252ee0bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:49.217656 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1367/2721] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:49.220091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:49.232120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118bfe-c905-4161-85aa-07330d0f9d80 service nova] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:49.237732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:49.238593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:49.239225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:49.239530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:49.240222 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:49.242371 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:49.275595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1355/2722] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:49:49 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:49.276892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e98f2-6778-4c80-9d56-fb2252ee0bc9 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:49.287745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e98f2-6778-4c80-9d56-fb2252ee0bc9 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:49.350343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:49.350968 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:49.351240 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:49.351504 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:49.352164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:49.354122 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:49.364072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:49.378847 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1368/2723] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:49:49 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:49:49.382885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:49.394216 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0964acb-17d0-4b8d-9702-840778501c15 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:49.399127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:49.399869 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:49.400150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:49.400414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:49.401315 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:49.402921 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:49.414894 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1356/2724] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:49.419180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:49.430743 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cfbea-e1f1-4db0-adb9-210d66817aad admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:49.435930 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:49.436601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:49.436903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:49.437182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] PATH_INFO: `/v3/projects/60f858018f734a1cba34dec09aec0f1a/users/44c09cbf366c4157a05471bef9491e61/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:49.453864 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f858018f734a1cba34dec09aec0f1a, user_id=44c09cbf366c4157a05471bef9491e61, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:49.456707 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:49.471684 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1369/2725] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:49 2022] PUT /identity/v3/projects/60f858018f734a1cba34dec09aec0f1a/users/44c09cbf366c4157a05471bef9491e61/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:49.475995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:49.510578 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0b249-43be-4737-b168-966e210a71e4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:49.581329 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:49.581975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:49.582215 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:49.582655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:49.583312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:49.584948 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:49.603960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1357/2726] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:49:49.604100 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:49.614038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479ee1-6eee-4c82-ad61-a1587d429fac admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:49.619191 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c2c77-da4d-4b22-99ca-8a055c549154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:49.621397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c2c77-da4d-4b22-99ca-8a055c549154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:49.621397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c2c77-da4d-4b22-99ca-8a055c549154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:49.621397 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c2c77-da4d-4b22-99ca-8a055c549154 admin admin] PATH_INFO: `/v3/projects/60f858018f734a1cba34dec09aec0f1a/users/44c09cbf366c4157a05471bef9491e61/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:49.624104 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c2c77-da4d-4b22-99ca-8a055c549154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f858018f734a1cba34dec09aec0f1a, user_id=44c09cbf366c4157a05471bef9491e61, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:49.626732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c2c77-da4d-4b22-99ca-8a055c549154 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:49.638113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1370/2727] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:49:49 2022] PUT /identity/v3/projects/60f858018f734a1cba34dec09aec0f1a/users/44c09cbf366c4157a05471bef9491e61/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:49:54.004038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:54.005801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:54.043931 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:49:54.207476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2955b-5aba-46f1-a0d5-9afd90798dd6 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:54.363648 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:54.364389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:54.364778 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:54.365079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:54.370860 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:54.388333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:54.390461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:54.411664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1358/2728] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:49:54.478541 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7404f49-3c31-432d-96fa-729dca7c484d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:54.479052 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7404f49-3c31-432d-96fa-729dca7c484d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:54.479152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7404f49-3c31-432d-96fa-729dca7c484d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:54.521622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-b7404f49-3c31-432d-96fa-729dca7c484d None None] MFA Rules not processed for user `44c09cbf366c4157a05471bef9491e61`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:49:54.523108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-b7404f49-3c31-432d-96fa-729dca7c484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:49:54.612807 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1371/2729] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:49:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:49:54.627746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:49:54.661386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc162bdf-a8c5-459b-9691-d71c0278fb8a service nova] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:49:54.682910 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:49:54.683655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:49:54.686065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:49:54.686358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:49:54.804912 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:49:54.806534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:49:54.840038 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1359/2730] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:49:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:50:04.613030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:INSERT=4,keystone:SELECT=51 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:50:04.823369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=84 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:50:16.911871 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138cae4-1da7-4384-8fc4-7ce23e2d64ef None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:50:16.912427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138cae4-1da7-4384-8fc4-7ce23e2d64ef None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:50:16.912427 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138cae4-1da7-4384-8fc4-7ce23e2d64ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:50:16.932245 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f138cae4-1da7-4384-8fc4-7ce23e2d64ef None None] MFA Rules not processed for user `bb6d35f57b804ba5afa4ab86c46cd7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:50:16.933470 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f138cae4-1da7-4384-8fc4-7ce23e2d64ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:50:16.941152 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1372/2731] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:50:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:50:26.926652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:50:35.932899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:50:35.992375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1880198-a05f-46a9-b4a6-545a688e31d8 service nova] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:50:35.996889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063519b-3e9f-4c34-aae3-def4d98d015e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:50:35.997506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063519b-3e9f-4c34-aae3-def4d98d015e service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:50:35.997786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063519b-3e9f-4c34-aae3-def4d98d015e service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:50:35.998057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063519b-3e9f-4c34-aae3-def4d98d015e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:50:36.008875 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063519b-3e9f-4c34-aae3-def4d98d015e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:50:36.010348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063519b-3e9f-4c34-aae3-def4d98d015e service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:50:36.020492 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1360/2732] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:50:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:50:46.016300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=26 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:51:01.950179 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138cae4-1da7-4384-8fc4-7ce23e2d64ef None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:51:01.967858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138cae4-1da7-4384-8fc4-7ce23e2d64ef None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:51:01.974268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:51:01.974906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:51:01.975204 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:51:01.976024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:51:01.978645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:51:01.995349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:51:01.997034 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:51:02.018290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1373/2733] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:51:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:51:10.926833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4c08-e78f-4ee0-a14b-828b4f8792c1 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:51:10.927362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4c08-e78f-4ee0-a14b-828b4f8792c1 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:51:10.927477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4c08-e78f-4ee0-a14b-828b4f8792c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:51:10.948421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-0e2c4c08-e78f-4ee0-a14b-828b4f8792c1 None None] MFA Rules not processed for user `ed76a574d2fc4bd8a1f1176d0fef86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:51:10.949627 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-0e2c4c08-e78f-4ee0-a14b-828b4f8792c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:51:11.009036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1361/2734] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:51:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:51:12.004456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:51:21.001110 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=25 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:51:30.550275 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:51:30.552234 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:51:30.625398 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe391e47-e414-43b9-90ac-c098d789140b service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:51:30.631267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9b4c-8148-4e51-868b-448c870ce8f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:51:30.631902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9b4c-8148-4e51-868b-448c870ce8f6 service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:51:30.632176 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9b4c-8148-4e51-868b-448c870ce8f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:51:30.632429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9b4c-8148-4e51-868b-448c870ce8f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:51:30.634341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-791f9b4c-8148-4e51-868b-448c870ce8f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:51:30.646502 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9b4c-8148-4e51-868b-448c870ce8f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:51:30.648140 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9b4c-8148-4e51-868b-448c870ce8f6 service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:51:30.657916 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1374/2735] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:51:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:51:40.654016 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=32 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:52:44.871301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4c08-e78f-4ee0-a14b-828b4f8792c1 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:52:44.884638 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4c08-e78f-4ee0-a14b-828b4f8792c1 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:52:44.889978 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4f91d-eedb-44d3-b6ab-ef637077ffb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:52:44.890577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4f91d-eedb-44d3-b6ab-ef637077ffb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:52:44.890848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4f91d-eedb-44d3-b6ab-ef637077ffb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:52:44.891117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4f91d-eedb-44d3-b6ab-ef637077ffb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:52:44.892961 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-63b4f91d-eedb-44d3-b6ab-ef637077ffb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:52:44.904332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4f91d-eedb-44d3-b6ab-ef637077ffb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:52:44.905628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4f91d-eedb-44d3-b6ab-ef637077ffb0 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:52:44.914631 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1362/2736] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:52:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:52:46.027566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdef51-dfd1-4a36-877f-5e02dec5a581 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:52:46.028103 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdef51-dfd1-4a36-877f-5e02dec5a581 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:52:46.028210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdef51-dfd1-4a36-877f-5e02dec5a581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:52:46.048487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-25cdef51-dfd1-4a36-877f-5e02dec5a581 None None] MFA Rules not processed for user `ed76a574d2fc4bd8a1f1176d0fef86e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:52:46.049716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-25cdef51-dfd1-4a36-877f-5e02dec5a581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:52:46.057946 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1375/2737] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:52:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:52:54.911290 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=6 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:52:55.170067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75155f-fbcf-4f87-9dd6-b694ec4397a4 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:52:55.170612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75155f-fbcf-4f87-9dd6-b694ec4397a4 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:52:55.171011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75155f-fbcf-4f87-9dd6-b694ec4397a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:52:55.190856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4e75155f-fbcf-4f87-9dd6-b694ec4397a4 None None] MFA Rules not processed for user `bb6d35f57b804ba5afa4ab86c46cd7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:52:55.192107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4e75155f-fbcf-4f87-9dd6-b694ec4397a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:52:55.199450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1363/2738] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:52:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:52:56.043101 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:53:05.185607 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:53:13.721733 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdef51-dfd1-4a36-877f-5e02dec5a581 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:53:13.733510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdef51-dfd1-4a36-877f-5e02dec5a581 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:53:13.740116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:53:13.740981 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:53:13.741316 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:53:13.741674 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:53:13.744677 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:53:13.762843 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:53:13.764438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:53:13.776713 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1376/2739] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:53:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:53:13.957349 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75155f-fbcf-4f87-9dd6-b694ec4397a4 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:53:13.968962 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75155f-fbcf-4f87-9dd6-b694ec4397a4 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:53:13.973299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:53:13.973908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:53:13.974187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:53:13.974462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:53:13.976366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:53:13.991597 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:53:13.993031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:53:14.005036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1364/2740] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:53:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:53:23.771627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:53:23.999437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=6 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:22.570073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:22.571693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:22.583046 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adb412-5f61-4a2f-9986-edd470aaa501 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:22.587279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:22.587923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:22.588239 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:22.588510 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:22.590638 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:22.662012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:22.663467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:22.674128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1377/2741] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:25.775476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:25.787109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1a969-8df7-4eff-9031-2b1825b91acf service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:25.793148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:25.793798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:25.794087 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:25.794461 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:25.796606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:25.808886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:25.810463 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:25.821309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1365/2742] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 05:54:25.840511 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:25.842472 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:25.856958 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b2fbb-fee0-4905-9d51-483e307b1c94 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:25.862268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:25.863175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:25.863693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:25.863861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] PATH_INFO: `/v3/users/ed76a574d2fc4bd8a1f1176d0fef86e2` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:25.865297 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed76a574d2fc4bd8a1f1176d0fef86e2)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:25.867306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:25.917811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed76a574d2fc4bd8a1f1176d0fef86e2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:25.918466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed76a574d2fc4bd8a1f1176d0fef86e2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:25.928561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed76a574d2fc4bd8a1f1176d0fef86e2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:25.929288 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed76a574d2fc4bd8a1f1176d0fef86e2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:25.943357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-04997cc2-bd88-4ece-b5ba-eb9d8a29140d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed76a574d2fc4bd8a1f1176d0fef86e2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:25.962375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1378/2743] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:25 2022] DELETE /identity/v3/users/ed76a574d2fc4bd8a1f1176d0fef86e2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:26.143280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.153871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a8d13-2c78-49bf-8ee0-60a162de6f72 service placement] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.213372 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.214687 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.215075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.215464 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] PATH_INFO: `/v3/projects/fae948b0baa04f2eb07b2aee2785dcdd` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.217489 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae948b0baa04f2eb07b2aee2785dcdd)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:26.220563 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:26.271929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae948b0baa04f2eb07b2aee2785dcdd', 'request_id': 'req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:26.278311 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae948b0baa04f2eb07b2aee2785dcdd', 'request_id': 'req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:26.285380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae948b0baa04f2eb07b2aee2785dcdd', 'request_id': 'req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:26.301078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] The token cache is being invalidate because project fae948b0baa04f2eb07b2aee2785dcdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:26.301359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:26.302081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-d9b2d2b7-aa22-4722-be4a-b116427d8ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:26.304026 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1366/2744] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:26 2022] DELETE /identity/v3/projects/fae948b0baa04f2eb07b2aee2785dcdd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:26.309436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95866730-4541-46ec-998f-c4d1fc47c9a3 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.309719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95866730-4541-46ec-998f-c4d1fc47c9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.309993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95866730-4541-46ec-998f-c4d1fc47c9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.329612 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-95866730-4541-46ec-998f-c4d1fc47c9a3 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:26.330795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-95866730-4541-46ec-998f-c4d1fc47c9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:26.393419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1379/2745] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:54:26.405084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa51fc-5f3f-4ba4-a0b4-9da55f1bc0cb None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.405364 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa51fc-5f3f-4ba4-a0b4-9da55f1bc0cb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.405633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa51fc-5f3f-4ba4-a0b4-9da55f1bc0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.426534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-d9aa51fc-5f3f-4ba4-a0b4-9da55f1bc0cb None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:26.427766 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d9aa51fc-5f3f-4ba4-a0b4-9da55f1bc0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:26.436506 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1367/2746] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:26.440913 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95866730-4541-46ec-998f-c4d1fc47c9a3 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.451424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95866730-4541-46ec-998f-c4d1fc47c9a3 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.456690 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.457328 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.457604 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.457872 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.458499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:54:26.458799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:26.460494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:26.462054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:26.473927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1380/2747] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:54:26.479094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa51fc-5f3f-4ba4-a0b4-9da55f1bc0cb None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.493787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa51fc-5f3f-4ba4-a0b4-9da55f1bc0cb None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.499086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.499725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.500004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.500270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.500909 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:26.502810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:26.532229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1368/2748] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:26 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:54:26.536498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.547106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff64daf-63da-4fde-a0d2-521b2a09aea5 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.607108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.607753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.608051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.608320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.608948 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:26.611022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:26.637538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1381/2749] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:26 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:54:26.643872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.659300 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960a3e8-ea86-47f6-8110-a41cba888b32 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.666078 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.666716 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.666995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.667265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.667938 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:26.669579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:26.680414 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1369/2750] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:26.684719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.695469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebd2c5-acf6-40b7-9a08-f4e090cd757c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.701439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f51f3-6cec-4a07-af36-e3eae8dc0974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.702184 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f51f3-6cec-4a07-af36-e3eae8dc0974 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.702484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f51f3-6cec-4a07-af36-e3eae8dc0974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.702716 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f51f3-6cec-4a07-af36-e3eae8dc0974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.703404 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f51f3-6cec-4a07-af36-e3eae8dc0974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:26.705026 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f51f3-6cec-4a07-af36-e3eae8dc0974 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:26.716827 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1382/2751] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:26.722231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.733434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de3441-536d-433a-9686-1b4b37125ba4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.739079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.739781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.740059 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.740326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] PATH_INFO: `/v3/projects/53eb4064d1184408b94fdc08bb5aa359/users/5c9dac0290b44af9bad13536d7383aa9/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.755897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53eb4064d1184408b94fdc08bb5aa359, user_id=5c9dac0290b44af9bad13536d7383aa9, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:26.758828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:26.775840 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1370/2752] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:26 2022] PUT /identity/v3/projects/53eb4064d1184408b94fdc08bb5aa359/users/5c9dac0290b44af9bad13536d7383aa9/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:26.786330 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e7a30-6b8f-4912-b5de-79df69d03733 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.786614 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e7a30-6b8f-4912-b5de-79df69d03733 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.786881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e7a30-6b8f-4912-b5de-79df69d03733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.815062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-584e7a30-6b8f-4912-b5de-79df69d03733 None None] MFA Rules not processed for user `5c9dac0290b44af9bad13536d7383aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:26.816315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-584e7a30-6b8f-4912-b5de-79df69d03733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:26.880646 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1383/2753] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:26.894798 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:26.896807 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:26.918129 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:26.980133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557551f-fb68-48f8-ace8-15d56fc5af12 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:26.986882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a5a3b-de14-487b-85b4-bd810fae5f3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:26.987490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a5a3b-de14-487b-85b4-bd810fae5f3d service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:26.987803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a5a3b-de14-487b-85b4-bd810fae5f3d service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:26.988082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a5a3b-de14-487b-85b4-bd810fae5f3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:26.999862 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a5a3b-de14-487b-85b4-bd810fae5f3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:27.002258 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a5a3b-de14-487b-85b4-bd810fae5f3d service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:27.013904 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1371/2754] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:27.251599 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e7a30-6b8f-4912-b5de-79df69d03733 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:27.316993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e7a30-6b8f-4912-b5de-79df69d03733 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:27.322824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:27.323462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:27.323988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:27.324267 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] PATH_INFO: `/v3/users/5c9dac0290b44af9bad13536d7383aa9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:27.325648 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9dac0290b44af9bad13536d7383aa9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:27.327594 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:27.330136 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:27.377251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9dac0290b44af9bad13536d7383aa9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:27.377883 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9dac0290b44af9bad13536d7383aa9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:27.384787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9dac0290b44af9bad13536d7383aa9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:27.385518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9dac0290b44af9bad13536d7383aa9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:27.395309 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9dac0290b44af9bad13536d7383aa9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:27.420445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1384/2755] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:27 2022] DELETE /identity/v3/users/5c9dac0290b44af9bad13536d7383aa9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:27.425022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:27.425333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:27.425593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:27.445187 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:27.446370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:27.510035 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1372/2756] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:54:27.520094 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:27.521590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:27.582144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87f30-acb8-4379-8bd9-ece724fa3648 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:27.586037 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e01b9e-e8d7-4373-b401-d58ecb7d397a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:27.586643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e01b9e-e8d7-4373-b401-d58ecb7d397a service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:27.586923 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e01b9e-e8d7-4373-b401-d58ecb7d397a service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:27.587185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e01b9e-e8d7-4373-b401-d58ecb7d397a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:27.596906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e01b9e-e8d7-4373-b401-d58ecb7d397a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:27.598428 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e01b9e-e8d7-4373-b401-d58ecb7d397a service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:27.607992 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1385/2757] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:27.900602 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:27.903355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:27.918015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb5258-45da-4be2-9e8b-dfe7d03240b0 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:27.924150 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:27.924827 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:27.925143 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:27.925361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] PATH_INFO: `/v3/projects/53eb4064d1184408b94fdc08bb5aa359` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:27.926541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=53eb4064d1184408b94fdc08bb5aa359)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:27.928615 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:27.979624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53eb4064d1184408b94fdc08bb5aa359', 'request_id': 'req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:27.985822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53eb4064d1184408b94fdc08bb5aa359', 'request_id': 'req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:27.992321 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53eb4064d1184408b94fdc08bb5aa359', 'request_id': 'req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:28.007060 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] The token cache is being invalidate because project 53eb4064d1184408b94fdc08bb5aa359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:28.007345 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:28.008015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-28c1fd3b-b862-4f05-a01a-c57147d4c0ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:28.009865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1373/2758] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:27 2022] DELETE /identity/v3/projects/53eb4064d1184408b94fdc08bb5aa359 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:28.016654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fa024-408e-4796-925f-6e06b11c6086 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.017155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fa024-408e-4796-925f-6e06b11c6086 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.017254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fa024-408e-4796-925f-6e06b11c6086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.042210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-8b6fa024-408e-4796-925f-6e06b11c6086 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:28.043436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-8b6fa024-408e-4796-925f-6e06b11c6086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:28.109466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1386/2759] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:28.118968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd824-95b9-4291-879a-0260877cd41e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.119253 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd824-95b9-4291-879a-0260877cd41e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.119526 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd824-95b9-4291-879a-0260877cd41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.141966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f3ccd824-95b9-4291-879a-0260877cd41e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:28.143195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f3ccd824-95b9-4291-879a-0260877cd41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:28.153539 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1374/2760] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:28.158153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fa024-408e-4796-925f-6e06b11c6086 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.168769 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fa024-408e-4796-925f-6e06b11c6086 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.174294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.174888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.175859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.176131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.176777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:54:28.177076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.178702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.197740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1387/2761] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:54:28.204701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccd824-95b9-4291-879a-0260877cd41e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.221267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccd824-95b9-4291-879a-0260877cd41e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.229865 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.230417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.230696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.231050 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.231714 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.233725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.249177 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:28.263833 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1375/2762] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:28 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:28.268110 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.278451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d8eb7-3300-4379-8788-99c6c4aa1f40 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.287061 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:28.339695 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.340308 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.340590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.340859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.341494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.343443 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.364310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1388/2763] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:28 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:54:28.368582 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.379695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe15060-5932-4c07-8803-b687c9fe51c1 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.385713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.386797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.387268 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.387557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.388266 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.390003 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.401151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1376/2764] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:28.405720 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.419537 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b9d26-52b1-497a-82a3-9d2fadede3cb admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.425040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81231d7-55ad-4942-9f91-135622195ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.425644 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81231d7-55ad-4942-9f91-135622195ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.425919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81231d7-55ad-4942-9f91-135622195ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.426182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81231d7-55ad-4942-9f91-135622195ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.426820 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81231d7-55ad-4942-9f91-135622195ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.428467 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81231d7-55ad-4942-9f91-135622195ecb admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.439830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1389/2765] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:28.444206 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.455121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3e72b-ac48-47fc-a28d-8cd7cffe630a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.461043 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.461737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.462016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.462286 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] PATH_INFO: `/v3/projects/451171ba151c4a0091bb1307c0cbc334/users/907dcd3583f74912b7a7cba9544f7ca2/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.479294 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] RBAC: Authorizing `identity:create_grant(project_id=451171ba151c4a0091bb1307c0cbc334, user_id=907dcd3583f74912b7a7cba9544f7ca2, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.482265 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.499700 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1377/2766] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:28 2022] PUT /identity/v3/projects/451171ba151c4a0091bb1307c0cbc334/users/907dcd3583f74912b7a7cba9544f7ca2/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:54:28.511455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecc2d1-8bb9-4d34-be47-2374ab87cbcc None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.511778 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecc2d1-8bb9-4d34-be47-2374ab87cbcc None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.512049 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecc2d1-8bb9-4d34-be47-2374ab87cbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.543544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-f4ecc2d1-8bb9-4d34-be47-2374ab87cbcc None None] MFA Rules not processed for user `907dcd3583f74912b7a7cba9544f7ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:28.545007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f4ecc2d1-8bb9-4d34-be47-2374ab87cbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:28.612769 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1390/2767] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:28.617369 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.688496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bb6fb-9b4e-4192-ab22-a5078bac224b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.694507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.695118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.695396 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.695695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.696320 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.698304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.729555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1378/2768] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:28 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:28.733805 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecc2d1-8bb9-4d34-be47-2374ab87cbcc None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.745543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecc2d1-8bb9-4d34-be47-2374ab87cbcc None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.814806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.815431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.815753 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.816040 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.816657 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.818590 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.841256 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1391/2769] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:28 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:28.845986 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.858204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750b81-d112-4d95-a658-f5d03665042e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.863970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.864570 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.864842 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.865127 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.865752 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.867352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.878512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1379/2770] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:28.883346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.894097 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebba894-dfa9-4593-8f13-db20e8071201 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:28.901112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:28.901795 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:28.902079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:28.902355 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] PATH_INFO: `/v3/projects/17bcc1c69be24faa9cbd16f4c120eabf/users/97bdf51293fe449abcd20c5afb2e9cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:28.905738 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:28.919466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bcc1c69be24faa9cbd16f4c120eabf, user_id=97bdf51293fe449abcd20c5afb2e9cbf, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:28.922180 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:28.936419 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1392/2771] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:28 2022] PUT /identity/v3/projects/17bcc1c69be24faa9cbd16f4c120eabf/users/97bdf51293fe449abcd20c5afb2e9cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:28.940647 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:28.951775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82ca6c-e42e-4e25-a77a-f46bb284b29a admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:29.021593 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904af43-7093-4210-b854-d3ab4acf5dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:29.022112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904af43-7093-4210-b854-d3ab4acf5dce admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:29.022387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904af43-7093-4210-b854-d3ab4acf5dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:29.022655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904af43-7093-4210-b854-d3ab4acf5dce admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:29.023280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904af43-7093-4210-b854-d3ab4acf5dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:29.024923 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904af43-7093-4210-b854-d3ab4acf5dce admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:29.035105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1380/2772] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:29.041393 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:29.052642 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b048a5-3629-47e0-adbb-9c8feef9b382 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:29.058118 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:29.058762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:29.059072 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:29.059348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] PATH_INFO: `/v3/projects/17bcc1c69be24faa9cbd16f4c120eabf/users/97bdf51293fe449abcd20c5afb2e9cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:29.064804 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bcc1c69be24faa9cbd16f4c120eabf, user_id=97bdf51293fe449abcd20c5afb2e9cbf, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:29.067446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:29.077956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1393/2773] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:29 2022] PUT /identity/v3/projects/17bcc1c69be24faa9cbd16f4c120eabf/users/97bdf51293fe449abcd20c5afb2e9cbf/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:29.089613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8a8fb-7ab8-4738-b012-791dfefa5c7d None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:29.090148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8a8fb-7ab8-4738-b012-791dfefa5c7d None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:29.090504 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8a8fb-7ab8-4738-b012-791dfefa5c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:29.123628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-dfc8a8fb-7ab8-4738-b012-791dfefa5c7d None None] MFA Rules not processed for user `97bdf51293fe449abcd20c5afb2e9cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:29.124995 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-dfc8a8fb-7ab8-4738-b012-791dfefa5c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:29.142919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:29.221046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1381/2774] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:29.237829 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:29.240569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:29.302862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bdc62-fc46-4e1c-ba73-d2a69be45018 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:29.307293 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632f34b-4bd0-405e-b240-1d43700d05e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:29.307920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632f34b-4bd0-405e-b240-1d43700d05e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:29.308197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632f34b-4bd0-405e-b240-1d43700d05e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:29.308463 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632f34b-4bd0-405e-b240-1d43700d05e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:29.318426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632f34b-4bd0-405e-b240-1d43700d05e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:29.319899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632f34b-4bd0-405e-b240-1d43700d05e3 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:29.332670 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1394/2775] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:29.364180 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8a8fb-7ab8-4738-b012-791dfefa5c7d None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:29.427051 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8a8fb-7ab8-4738-b012-791dfefa5c7d None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:29.432133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:29.432821 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:29.433096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:29.433366 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] PATH_INFO: `/v3/users/907dcd3583f74912b7a7cba9544f7ca2` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:29.434520 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=907dcd3583f74912b7a7cba9544f7ca2)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:29.436500 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:29.491192 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '907dcd3583f74912b7a7cba9544f7ca2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:29.491882 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '907dcd3583f74912b7a7cba9544f7ca2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:29.492553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '907dcd3583f74912b7a7cba9544f7ca2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:29.501666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '907dcd3583f74912b7a7cba9544f7ca2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:29.511852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '907dcd3583f74912b7a7cba9544f7ca2'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:29.535115 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1382/2776] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:29 2022] DELETE /identity/v3/users/907dcd3583f74912b7a7cba9544f7ca2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:29.539900 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:29.540209 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:29.540520 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:29.560169 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:29.561306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:29.622742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1395/2777] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:29.634423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:29.636106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:29.672344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:29.703348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b7656-d18c-433b-8350-3f204da52055 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:29.707441 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:29.708088 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:29.708362 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:29.708628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:29.718274 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:29.719737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:29.729434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1383/2778] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:30.014759 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:30.016229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:30.029004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95945fa1-27c5-42af-86ed-e57ad02bdbe0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:30.034461 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:30.035142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:30.035421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:30.035732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] PATH_INFO: `/v3/projects/451171ba151c4a0091bb1307c0cbc334` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:30.036998 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=451171ba151c4a0091bb1307c0cbc334)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:30.039004 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:30.088569 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '451171ba151c4a0091bb1307c0cbc334', 'request_id': 'req-dd097027-977f-45b8-b810-4d14e369f815'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.094302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '451171ba151c4a0091bb1307c0cbc334', 'request_id': 'req-dd097027-977f-45b8-b810-4d14e369f815'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.100585 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '451171ba151c4a0091bb1307c0cbc334', 'request_id': 'req-dd097027-977f-45b8-b810-4d14e369f815'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.110582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:30.115481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] The token cache is being invalidate because project 451171ba151c4a0091bb1307c0cbc334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:30.115799 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.116484 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.118451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1396/2779] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:30 2022] DELETE /identity/v3/projects/451171ba151c4a0091bb1307c0cbc334 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:30.122856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:30.132598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a7256-a44a-4e65-8d2f-07b79ad4229f service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:30.196082 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:30.196716 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:30.196993 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:30.197287 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] PATH_INFO: `/v3/users/97bdf51293fe449abcd20c5afb2e9cbf` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:30.198691 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97bdf51293fe449abcd20c5afb2e9cbf)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:30.200664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:30.255361 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97bdf51293fe449abcd20c5afb2e9cbf'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.256182 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97bdf51293fe449abcd20c5afb2e9cbf'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.256857 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97bdf51293fe449abcd20c5afb2e9cbf'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.265473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97bdf51293fe449abcd20c5afb2e9cbf'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.272352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b6c65238-6b21-4cbe-8d5f-d2a04e6200a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97bdf51293fe449abcd20c5afb2e9cbf'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.288848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1384/2780] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:30 2022] DELETE /identity/v3/users/97bdf51293fe449abcd20c5afb2e9cbf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:30.582886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:30.584508 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:30.598833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097027-977f-45b8-b810-4d14e369f815 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:30.658128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:30.658774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:30.659035 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:30.659329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] PATH_INFO: `/v3/projects/17bcc1c69be24faa9cbd16f4c120eabf` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:30.660494 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bcc1c69be24faa9cbd16f4c120eabf)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:30.662436 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:30.711130 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bcc1c69be24faa9cbd16f4c120eabf', 'request_id': 'req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.717622 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bcc1c69be24faa9cbd16f4c120eabf', 'request_id': 'req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.723963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bcc1c69be24faa9cbd16f4c120eabf', 'request_id': 'req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.739976 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] The token cache is being invalidate because project 17bcc1c69be24faa9cbd16f4c120eabf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:30.740290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.740945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-94be5ed2-ff53-4d2e-a04c-0c33f8defcf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:30.742437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1397/2781] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:30 2022] DELETE /identity/v3/projects/17bcc1c69be24faa9cbd16f4c120eabf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:30.747969 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66043bbb-dcf0-4938-b619-9057df7198a9 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:30.748263 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66043bbb-dcf0-4938-b619-9057df7198a9 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:30.748534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66043bbb-dcf0-4938-b619-9057df7198a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:30.768685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-66043bbb-dcf0-4938-b619-9057df7198a9 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:30.769897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-66043bbb-dcf0-4938-b619-9057df7198a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:30.826957 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:30.844719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1385/2782] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:30.852691 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf313f28-5154-4d8f-bce4-f6704e0ad190 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:30.852888 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf313f28-5154-4d8f-bce4-f6704e0ad190 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:30.853197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf313f28-5154-4d8f-bce4-f6704e0ad190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:30.873318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bf313f28-5154-4d8f-bce4-f6704e0ad190 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:30.874531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bf313f28-5154-4d8f-bce4-f6704e0ad190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:30.882863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1398/2783] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:30.887505 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66043bbb-dcf0-4938-b619-9057df7198a9 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:30.899633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66043bbb-dcf0-4938-b619-9057df7198a9 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:30.905353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:30.905977 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:30.906249 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:30.906512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:30.907169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:54:30.907470 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:30.909099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:30.922029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1386/2784] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:54:30.926764 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf313f28-5154-4d8f-bce4-f6704e0ad190 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:30.936773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf313f28-5154-4d8f-bce4-f6704e0ad190 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:30.941506 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:30.942111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:30.942386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:30.942650 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:30.943271 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:30.945215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:30.976816 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1399/2785] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:30 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:30.983029 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:30.997260 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bf9e1-abf7-4dff-8e55-80b0f816b8ec admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.053437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.054047 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.054325 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.054595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.055219 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.057217 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.082070 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1387/2786] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:30 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:31.086626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.097963 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7209-6778-42a7-ae5e-760dbdd22bca admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.103975 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.104595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.104874 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.105143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.105788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.107431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.117499 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1400/2787] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:31.122303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.137083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242991d-4d89-44e9-abc8-89829fbb984c admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.143006 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfd33a-6308-4414-8a22-e47f11a21972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.143611 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd33a-6308-4414-8a22-e47f11a21972 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.144072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd33a-6308-4414-8a22-e47f11a21972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.144342 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd33a-6308-4414-8a22-e47f11a21972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.144994 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfd33a-6308-4414-8a22-e47f11a21972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.146612 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfd33a-6308-4414-8a22-e47f11a21972 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.158425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1388/2788] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:31.165113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.178062 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d5998-3e3d-44ef-a42c-e8cb58276bb9 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.187044 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.188127 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.188567 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.188987 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] PATH_INFO: `/v3/projects/dbdba2b880084b839132e55c9f309e22/users/57a1bb57dbb540cca323abc959e2c27e/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.208482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdba2b880084b839132e55c9f309e22, user_id=57a1bb57dbb540cca323abc959e2c27e, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.211178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.228059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1401/2789] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:31 2022] PUT /identity/v3/projects/dbdba2b880084b839132e55c9f309e22/users/57a1bb57dbb540cca323abc959e2c27e/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:31.239887 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542bbf1-26ee-4966-ab74-93a87982ad08 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.240169 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542bbf1-26ee-4966-ab74-93a87982ad08 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.240436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542bbf1-26ee-4966-ab74-93a87982ad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.270152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-f542bbf1-26ee-4966-ab74-93a87982ad08 None None] MFA Rules not processed for user `57a1bb57dbb540cca323abc959e2c27e`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:31.271352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-f542bbf1-26ee-4966-ab74-93a87982ad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:31.343889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1389/2790] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:31.351114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.380431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:31.422149 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8381cf5-0c60-4e8d-aade-b241242a93e4 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.427322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.427950 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.428228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.428498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.429111 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.431014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.469254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1402/2791] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:31 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:31.473451 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542bbf1-26ee-4966-ab74-93a87982ad08 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.484656 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542bbf1-26ee-4966-ab74-93a87982ad08 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.535864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:31.546450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.547063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.547344 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.547640 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.548299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.550285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.572277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1390/2792] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:31 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:31.576566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.587025 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f61c3-a04d-4f25-adae-e22323ce69e9 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.592808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.593415 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.593686 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.593956 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.594595 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.596233 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.607474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1403/2793] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:31.611812 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.622142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e378b03-90b9-47b8-b5f1-992d0eb31ecd admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.626837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.627494 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.627803 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.628075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] PATH_INFO: `/v3/projects/64cd68c7b65940cb980c591b06268750/users/f79a0b87c4fc4ba8b499089ecb85a912/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.643357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64cd68c7b65940cb980c591b06268750, user_id=f79a0b87c4fc4ba8b499089ecb85a912, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.645974 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.664462 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1391/2794] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:31 2022] PUT /identity/v3/projects/64cd68c7b65940cb980c591b06268750/users/f79a0b87c4fc4ba8b499089ecb85a912/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:31.668572 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.679346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f7cb-85fe-4807-8adb-cebaabb26038 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.736868 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa45204-9eed-4809-8f36-ac97ee3fe5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.737474 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa45204-9eed-4809-8f36-ac97ee3fe5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.737746 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa45204-9eed-4809-8f36-ac97ee3fe5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.738010 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa45204-9eed-4809-8f36-ac97ee3fe5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.738643 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa45204-9eed-4809-8f36-ac97ee3fe5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.740279 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa45204-9eed-4809-8f36-ac97ee3fe5ea admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.750426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1404/2795] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:31.754697 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.764985 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7ce26-1860-4c87-8e4b-2a2faa754870 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:31.769587 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:31.770236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.770512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.770779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] PATH_INFO: `/v3/projects/64cd68c7b65940cb980c591b06268750/users/f79a0b87c4fc4ba8b499089ecb85a912/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.772713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=64cd68c7b65940cb980c591b06268750, user_id=f79a0b87c4fc4ba8b499089ecb85a912, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:31.775326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:31.785951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1392/2796] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:31 2022] PUT /identity/v3/projects/64cd68c7b65940cb980c591b06268750/users/f79a0b87c4fc4ba8b499089ecb85a912/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:31.796041 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:31.796324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:31.796601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:31.831145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] MFA Rules not processed for user `f79a0b87c4fc4ba8b499089ecb85a912`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:31.832876 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:31.919224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1405/2797] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:31.930964 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:31.932901 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:32.000285 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad3928-3d8a-478d-9117-fb859a792efc admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:32.006443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:32.007230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:32.007624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:32.008004 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:32.018255 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:32.019735 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:32.029374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1393/2798] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:32.046134 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:32.047538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:32.072313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:32.097889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:32.115182 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75769-b971-491c-a6ea-7caa440ce71a None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:32.119203 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7a706-f6cd-4fd4-ab2c-0b67e28c35a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:32.120577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7a706-f6cd-4fd4-ab2c-0b67e28c35a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:32.120850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7a706-f6cd-4fd4-ab2c-0b67e28c35a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:32.121126 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7a706-f6cd-4fd4-ab2c-0b67e28c35a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:32.123005 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-4be7a706-f6cd-4fd4-ab2c-0b67e28c35a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:32.133013 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7a706-f6cd-4fd4-ab2c-0b67e28c35a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:32.134441 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7a706-f6cd-4fd4-ab2c-0b67e28c35a7 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:32.143368 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1406/2799] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:32.163357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8584b-c560-48ad-88f6-aed87decb261 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:32.168621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:32.169259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:32.169534 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:32.169799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] PATH_INFO: `/v3/users/57a1bb57dbb540cca323abc959e2c27e` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:32.170906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a1bb57dbb540cca323abc959e2c27e)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:32.172850 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:32.228359 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a1bb57dbb540cca323abc959e2c27e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.229851 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a1bb57dbb540cca323abc959e2c27e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.230613 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a1bb57dbb540cca323abc959e2c27e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.232934 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:32.240214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a1bb57dbb540cca323abc959e2c27e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.248808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a1bb57dbb540cca323abc959e2c27e'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.266661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1394/2800] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:32 2022] DELETE /identity/v3/users/57a1bb57dbb540cca323abc959e2c27e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:32.271313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:32.271626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:32.271907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:32.293857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:32.295074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:32.361832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1407/2801] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:32.372436 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:32.373980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:32.433980 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a5199-2b9d-45b1-b66f-a93288e93aea admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:32.437686 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:32.438288 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:32.438579 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:32.438849 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:32.448802 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:32.450270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:32.460054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1395/2802] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:32.741629 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:32.743200 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:32.767939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de452ba6-7abb-4672-a2bd-7cbbc4ff37de None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:32.774067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:32.774744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:32.775048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:32.775322 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] PATH_INFO: `/v3/projects/dbdba2b880084b839132e55c9f309e22` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:32.777059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbdba2b880084b839132e55c9f309e22)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:32.779105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:32.827808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbdba2b880084b839132e55c9f309e22', 'request_id': 'req-f64ba030-2919-4cb8-bd08-e3b794c7d679'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.833563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbdba2b880084b839132e55c9f309e22', 'request_id': 'req-f64ba030-2919-4cb8-bd08-e3b794c7d679'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.840371 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbdba2b880084b839132e55c9f309e22', 'request_id': 'req-f64ba030-2919-4cb8-bd08-e3b794c7d679'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.853349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] The token cache is being invalidate because project dbdba2b880084b839132e55c9f309e22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:32.853631 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.854676 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.856290 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1408/2803] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:32 2022] DELETE /identity/v3/projects/dbdba2b880084b839132e55c9f309e22 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:32.862397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:32.877915 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc0da3-ad1b-42f5-903f-8e6ccc4f0752 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:32.940682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:32.941230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:32.941505 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:32.941771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] PATH_INFO: `/v3/users/f79a0b87c4fc4ba8b499089ecb85a912` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:32.942908 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79a0b87c4fc4ba8b499089ecb85a912)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:32.944877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:32.997577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79a0b87c4fc4ba8b499089ecb85a912'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.998254 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79a0b87c4fc4ba8b499089ecb85a912'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:32.998858 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79a0b87c4fc4ba8b499089ecb85a912'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.009786 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79a0b87c4fc4ba8b499089ecb85a912'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.017572 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-a3f5025a-3919-4999-99db-8937a425dbd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79a0b87c4fc4ba8b499089ecb85a912'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.024277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:33.038676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1396/2804] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:32 2022] DELETE /identity/v3/users/f79a0b87c4fc4ba8b499089ecb85a912 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:33.329639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:33.332009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:33.347834 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ba030-2919-4cb8-bd08-e3b794c7d679 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:33.395609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:33.410906 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:33.411549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:33.411863 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:33.412143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] PATH_INFO: `/v3/projects/64cd68c7b65940cb980c591b06268750` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:33.413300 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64cd68c7b65940cb980c591b06268750)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:33.415246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:33.474618 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64cd68c7b65940cb980c591b06268750', 'request_id': 'req-457c3088-e7b5-4513-a48b-c82e423582a2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.482038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64cd68c7b65940cb980c591b06268750', 'request_id': 'req-457c3088-e7b5-4513-a48b-c82e423582a2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.488543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64cd68c7b65940cb980c591b06268750', 'request_id': 'req-457c3088-e7b5-4513-a48b-c82e423582a2'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.501937 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] The token cache is being invalidate because project 64cd68c7b65940cb980c591b06268750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:33.502222 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.502832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-457c3088-e7b5-4513-a48b-c82e423582a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:33.504349 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1409/2805] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:33 2022] DELETE /identity/v3/projects/64cd68c7b65940cb980c591b06268750 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:54:33.509507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb07ad-1803-4418-9756-8f0e4c7021fa None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:33.510089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb07ad-1803-4418-9756-8f0e4c7021fa None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:33.510089 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb07ad-1803-4418-9756-8f0e4c7021fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:33.531609 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-62fb07ad-1803-4418-9756-8f0e4c7021fa None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:33.532854 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-62fb07ad-1803-4418-9756-8f0e4c7021fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:33.593713 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1397/2806] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:33.603338 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01bf44-87dd-48b9-97b0-fdc299eaf9f8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:33.603639 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01bf44-87dd-48b9-97b0-fdc299eaf9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:33.604260 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01bf44-87dd-48b9-97b0-fdc299eaf9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:33.751320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9d01bf44-87dd-48b9-97b0-fdc299eaf9f8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:33.752481 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9d01bf44-87dd-48b9-97b0-fdc299eaf9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:33.761482 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1410/2807] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:33.768339 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb07ad-1803-4418-9756-8f0e4c7021fa None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:33.781608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb07ad-1803-4418-9756-8f0e4c7021fa None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:33.787701 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:33.788348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:33.788628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:33.788899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:33.789595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:54:33.789890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:33.791567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:33.805130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1398/2808] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:54:33.809922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01bf44-87dd-48b9-97b0-fdc299eaf9f8 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:33.824445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01bf44-87dd-48b9-97b0-fdc299eaf9f8 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:33.831091 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:33.831736 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:33.832015 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:33.832280 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:33.832907 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:33.834811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:33.870514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1411/2809] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:33 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:33.875536 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:33.890486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3a174-98a9-4fe2-a188-258484cb4499 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:33.953058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:33.953673 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:33.953954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:33.954213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:33.954835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:33.956800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:33.987375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1399/2810] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:33 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:33.991723 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.003544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325a3fe-fa68-4acb-8cfa-64689aa1918c admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.008344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.008945 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.009221 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.009517 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.010361 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.012434 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.023403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1412/2811] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:34.027946 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.038149 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0154727-4dfb-4946-b7da-a0665605e3c9 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.043755 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea0711-c2a7-46ae-80af-3000e050e025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.044358 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea0711-c2a7-46ae-80af-3000e050e025 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.044635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea0711-c2a7-46ae-80af-3000e050e025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.044903 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea0711-c2a7-46ae-80af-3000e050e025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.045524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea0711-c2a7-46ae-80af-3000e050e025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.047117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea0711-c2a7-46ae-80af-3000e050e025 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.057877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1400/2812] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:34.064038 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.078892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13336d3-36fb-4e37-a853-9be446bc0ae8 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.086469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.087095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.087477 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.087802 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] PATH_INFO: `/v3/projects/72005ce1de394744a1fd79535c605dc3/users/7f1813d189e94e5282934309873569bd/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.103093 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=72005ce1de394744a1fd79535c605dc3, user_id=7f1813d189e94e5282934309873569bd, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.105896 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.124073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1413/2813] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:34 2022] PUT /identity/v3/projects/72005ce1de394744a1fd79535c605dc3/users/7f1813d189e94e5282934309873569bd/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:34.190176 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d9bab-9e96-4c89-ac46-0d83c350192f None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.190606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d9bab-9e96-4c89-ac46-0d83c350192f None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.191151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d9bab-9e96-4c89-ac46-0d83c350192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.230530 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-518d9bab-9e96-4c89-ac46-0d83c350192f None None] MFA Rules not processed for user `7f1813d189e94e5282934309873569bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:34.231670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-518d9bab-9e96-4c89-ac46-0d83c350192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:34.290511 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:34.316953 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1401/2814] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:34.322117 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.402830 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f41950-f9ee-446c-b96e-ea06b08332ff admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.407814 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.408515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.408997 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.409326 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.410081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.412031 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.436425 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:34.444893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1414/2815] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:34 2022] POST /identity/v3/projects => generated 401 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:34.449429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d9bab-9e96-4c89-ac46-0d83c350192f None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.460333 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d9bab-9e96-4c89-ac46-0d83c350192f None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.519097 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.519828 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.520011 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.520283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.520919 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.522837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.550749 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1402/2816] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:34 2022] POST /identity/v3/users => generated 307 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:34.551935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.562437 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e7a49-42d9-4aae-824f-7b690a172b2b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.567315 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.567952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.568229 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.568500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.569175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.570787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.580765 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1415/2817] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:34.584916 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.596041 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b2a62-95e4-405f-a576-8a37324cea8f admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.601305 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.601990 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.602269 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.602525 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] PATH_INFO: `/v3/projects/e276e9d206c94dfc85b0f7291a311a0e/users/248b6c8d98174f6daf1eb9d8d26b51df/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.618118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e276e9d206c94dfc85b0f7291a311a0e, user_id=248b6c8d98174f6daf1eb9d8d26b51df, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.621343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.636561 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1403/2818] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:34 2022] PUT /identity/v3/projects/e276e9d206c94dfc85b0f7291a311a0e/users/248b6c8d98174f6daf1eb9d8d26b51df/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:34.640623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.651249 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f2bdf-4b88-499a-96af-0fa640c4aa20 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.712186 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb6f83-d6c2-41b2-96cc-7ef3cf074a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.712790 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb6f83-d6c2-41b2-96cc-7ef3cf074a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.713073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb6f83-d6c2-41b2-96cc-7ef3cf074a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.713392 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb6f83-d6c2-41b2-96cc-7ef3cf074a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.714030 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb6f83-d6c2-41b2-96cc-7ef3cf074a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.716067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb6f83-d6c2-41b2-96cc-7ef3cf074a15 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.725726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1416/2819] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:34.730158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.742560 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a21c-50fb-4a33-9c6e-1d5b66fae981 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.748301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.748959 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.749243 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.749512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] PATH_INFO: `/v3/projects/e276e9d206c94dfc85b0f7291a311a0e/users/248b6c8d98174f6daf1eb9d8d26b51df/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.751480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e276e9d206c94dfc85b0f7291a311a0e, user_id=248b6c8d98174f6daf1eb9d8d26b51df, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.754157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.764663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1404/2820] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:34 2022] PUT /identity/v3/projects/e276e9d206c94dfc85b0f7291a311a0e/users/248b6c8d98174f6daf1eb9d8d26b51df/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:34.774914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76b124-ff89-4d18-a6a8-70da954fa192 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.775192 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76b124-ff89-4d18-a6a8-70da954fa192 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.775458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76b124-ff89-4d18-a6a8-70da954fa192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.803187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-0a76b124-ff89-4d18-a6a8-70da954fa192 None None] MFA Rules not processed for user `248b6c8d98174f6daf1eb9d8d26b51df`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:34.804344 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0a76b124-ff89-4d18-a6a8-70da954fa192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:34.878705 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1417/2821] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:34 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:34.889900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:34.892134 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:34.958645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3faac7-85ae-471e-9155-48a853f87bfa admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:34.962375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:34.962988 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:34.963272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:34.964080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:34.973332 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:34.974818 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:34.984592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1405/2822] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:34 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:35.068175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76b124-ff89-4d18-a6a8-70da954fa192 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:35.078627 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76b124-ff89-4d18-a6a8-70da954fa192 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:35.082521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb8de6-989d-4912-baee-7a66d5a31df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:35.083145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb8de6-989d-4912-baee-7a66d5a31df9 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:35.083431 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb8de6-989d-4912-baee-7a66d5a31df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:35.083718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb8de6-989d-4912-baee-7a66d5a31df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:35.143217 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb8de6-989d-4912-baee-7a66d5a31df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:35.144801 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb8de6-989d-4912-baee-7a66d5a31df9 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:35.155174 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1418/2823] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:35 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:39.318374 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:39.359091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:39.390126 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32c018-abeb-498d-a33a-9830973cff45 service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:39.395839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:39.396477 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:39.396762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:39.397032 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] PATH_INFO: `/v3/users/7f1813d189e94e5282934309873569bd` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:39.398201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1813d189e94e5282934309873569bd)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:39.400179 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:39.449914 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1813d189e94e5282934309873569bd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:39.450480 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1813d189e94e5282934309873569bd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:39.451122 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1813d189e94e5282934309873569bd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:39.459637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1813d189e94e5282934309873569bd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:39.467337 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1813d189e94e5282934309873569bd'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:39.485301 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1406/2824] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:39 2022] DELETE /identity/v3/users/7f1813d189e94e5282934309873569bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:39.491521 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:39.492039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:39.492197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:39.506218 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=100 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:39.512350 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:39.513561 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:39.577869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1419/2825] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:54:39.591553 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:39.593775 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:39.656081 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279b4f0-0811-44ab-86b2-8cb7e328ddbb admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:39.660498 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:39.661107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:39.661386 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:39.661655 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:39.671595 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:39.673118 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:39.682785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1407/2826] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:39.974810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:39.976408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:39.996986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac74b08-6be3-42ea-af28-d7f4720819ce None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:40.002741 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:40.003406 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:40.003724 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:40.003972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] PATH_INFO: `/v3/projects/72005ce1de394744a1fd79535c605dc3` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:40.005077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72005ce1de394744a1fd79535c605dc3)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:40.008960 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:40.057341 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72005ce1de394744a1fd79535c605dc3', 'request_id': 'req-35c43c1c-1e88-4501-adfd-4562ec76c902'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.062584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72005ce1de394744a1fd79535c605dc3', 'request_id': 'req-35c43c1c-1e88-4501-adfd-4562ec76c902'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.068566 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72005ce1de394744a1fd79535c605dc3', 'request_id': 'req-35c43c1c-1e88-4501-adfd-4562ec76c902'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.081532 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] The token cache is being invalidate because project 72005ce1de394744a1fd79535c605dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:40.081811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.082432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.083915 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1420/2827] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:39 2022] DELETE /identity/v3/projects/72005ce1de394744a1fd79535c605dc3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:40.088102 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:40.098107 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e568-eaac-40ae-9886-8e1dd8a997b7 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:40.155548 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:40.156201 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:40.156473 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:40.156740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] PATH_INFO: `/v3/users/248b6c8d98174f6daf1eb9d8d26b51df` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:40.158096 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=248b6c8d98174f6daf1eb9d8d26b51df)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:40.160054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:40.161495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:40.209661 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '248b6c8d98174f6daf1eb9d8d26b51df'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.210335 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '248b6c8d98174f6daf1eb9d8d26b51df'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.210976 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '248b6c8d98174f6daf1eb9d8d26b51df'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.220259 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '248b6c8d98174f6daf1eb9d8d26b51df'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.227360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-b1379a01-eb9f-430e-a731-25552347e439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '248b6c8d98174f6daf1eb9d8d26b51df'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.244670 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1408/2828] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:40 2022] DELETE /identity/v3/users/248b6c8d98174f6daf1eb9d8d26b51df => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:40.528678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:40.530194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:40.544228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c43c1c-1e88-4501-adfd-4562ec76c902 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:40.606986 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:40.607527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:40.607840 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:40.608112 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] PATH_INFO: `/v3/projects/e276e9d206c94dfc85b0f7291a311a0e` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:40.609346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e276e9d206c94dfc85b0f7291a311a0e)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:40.611849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:40.648982 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:40.663421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e276e9d206c94dfc85b0f7291a311a0e', 'request_id': 'req-7acb9445-f50f-4836-94fe-7b14b0521ab0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.669902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e276e9d206c94dfc85b0f7291a311a0e', 'request_id': 'req-7acb9445-f50f-4836-94fe-7b14b0521ab0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.676329 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e276e9d206c94dfc85b0f7291a311a0e', 'request_id': 'req-7acb9445-f50f-4836-94fe-7b14b0521ab0'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.694920 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] The token cache is being invalidate because project e276e9d206c94dfc85b0f7291a311a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:40.695314 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.696395 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-7acb9445-f50f-4836-94fe-7b14b0521ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:40.698806 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1421/2829] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:40 2022] DELETE /identity/v3/projects/e276e9d206c94dfc85b0f7291a311a0e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:54:40.704162 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416edf0-f644-4c4a-8161-030ced95e245 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:40.704637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416edf0-f644-4c4a-8161-030ced95e245 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:40.704800 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416edf0-f644-4c4a-8161-030ced95e245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:40.725867 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4416edf0-f644-4c4a-8161-030ced95e245 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:40.727072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4416edf0-f644-4c4a-8161-030ced95e245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:40.792394 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1409/2830] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:40.801760 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea2748-5726-4137-95ef-84c5f654ce29 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:40.802048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea2748-5726-4137-95ef-84c5f654ce29 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:40.802318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea2748-5726-4137-95ef-84c5f654ce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:40.826261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-9eea2748-5726-4137-95ef-84c5f654ce29 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:40.828187 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-9eea2748-5726-4137-95ef-84c5f654ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:40.837776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1422/2831] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:40.842594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416edf0-f644-4c4a-8161-030ced95e245 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:40.853405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416edf0-f644-4c4a-8161-030ced95e245 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:40.859434 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:40.860009 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:40.860486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:40.860598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:40.861235 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:54:40.861538 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:40.863135 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:40.876729 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1410/2832] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:54:40.881230 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea2748-5726-4137-95ef-84c5f654ce29 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:40.891048 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea2748-5726-4137-95ef-84c5f654ce29 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:40.896858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:40.897523 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:40.897810 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:40.898113 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:40.898749 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:40.901153 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:40.930028 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1423/2833] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:40 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:40.934437 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:40.944947 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb9b1e-520f-4ddb-8756-632abff50fcf admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.004626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.005487 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.005837 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.006162 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.007105 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.010229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.023748 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:41.035794 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1411/2834] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:40 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:41.039294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.049924 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7764c0-d925-4c24-8649-0ea5d62e518b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.055033 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.055674 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.055954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.056257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.056902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.058514 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.069294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1424/2835] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:41.075142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.086456 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86fe1e-bedd-4e27-8412-6d888e479462 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.093154 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c01af-f858-4104-97a8-ae5fde88b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.093762 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c01af-f858-4104-97a8-ae5fde88b681 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.094049 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c01af-f858-4104-97a8-ae5fde88b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.094312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c01af-f858-4104-97a8-ae5fde88b681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.095072 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c01af-f858-4104-97a8-ae5fde88b681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.096719 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c01af-f858-4104-97a8-ae5fde88b681 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.108112 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1412/2836] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:41.112645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.123241 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803b7a7-fd73-43d8-84e0-a03a1f38b7f6 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.129291 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.129939 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.130273 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.130548 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] PATH_INFO: `/v3/projects/5e95a012396b4e40ad7ee931586a5dc2/users/58ca8d9f6084412b969c11e522c23733/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.145626 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e95a012396b4e40ad7ee931586a5dc2, user_id=58ca8d9f6084412b969c11e522c23733, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.148302 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.164761 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1425/2837] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:41 2022] PUT /identity/v3/projects/5e95a012396b4e40ad7ee931586a5dc2/users/58ca8d9f6084412b969c11e522c23733/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:41.175481 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e90ab-8a3a-43df-8926-3bcdeb0ea3cb None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.175804 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e90ab-8a3a-43df-8926-3bcdeb0ea3cb None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.176079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e90ab-8a3a-43df-8926-3bcdeb0ea3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.206091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-5d0e90ab-8a3a-43df-8926-3bcdeb0ea3cb None None] MFA Rules not processed for user `58ca8d9f6084412b969c11e522c23733`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:41.207322 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5d0e90ab-8a3a-43df-8926-3bcdeb0ea3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:41.287586 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1413/2838] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:41.292895 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.362458 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb429e-36dc-409b-af36-63942d1c9d6b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.368386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.368919 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.369189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.369462 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] PATH_INFO: `/v3/projects` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.370082 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.372017 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.401518 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1426/2839] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:41 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:41.407331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e90ab-8a3a-43df-8926-3bcdeb0ea3cb None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.418733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e90ab-8a3a-43df-8926-3bcdeb0ea3cb None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.475271 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.475886 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.476158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.476423 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] PATH_INFO: `/v3/users` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.477054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.478954 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.502225 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1414/2840] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:41 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:54:41.509719 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.524185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73794c5-db48-421c-8ffa-f1847ff3462f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.529120 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.529740 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.529988 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.530246 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.530886 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.532543 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.543480 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1427/2841] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:41.548230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.558562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40796e5c-5eb5-472c-9156-65ed4df6ecfd admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.563323 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.564031 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.564309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.564628 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] PATH_INFO: `/v3/projects/b18bd6a37d984ac3b95eb58793679b7b/users/25a887c652214a8aa3113d1451dc09f7/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.580685 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18bd6a37d984ac3b95eb58793679b7b, user_id=25a887c652214a8aa3113d1451dc09f7, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.583331 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.599008 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1415/2842] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:41 2022] PUT /identity/v3/projects/b18bd6a37d984ac3b95eb58793679b7b/users/25a887c652214a8aa3113d1451dc09f7/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:41.604424 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.614455 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450e880-f45d-4870-8415-8010f8b8bd31 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.642892 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:41.671962 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a32a3d-df20-4c55-9ffb-bfd8b6a25f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.672563 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a32a3d-df20-4c55-9ffb-bfd8b6a25f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.672881 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a32a3d-df20-4c55-9ffb-bfd8b6a25f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.673144 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a32a3d-df20-4c55-9ffb-bfd8b6a25f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.673763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a32a3d-df20-4c55-9ffb-bfd8b6a25f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.675348 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a32a3d-df20-4c55-9ffb-bfd8b6a25f67 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.684911 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1428/2843] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:41.689348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.701773 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051556f6-ade2-48be-9685-3818882c87ac admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.709356 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.710067 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.710443 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.710796 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] PATH_INFO: `/v3/projects/b18bd6a37d984ac3b95eb58793679b7b/users/25a887c652214a8aa3113d1451dc09f7/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.714702 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18bd6a37d984ac3b95eb58793679b7b, user_id=25a887c652214a8aa3113d1451dc09f7, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.718971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.732476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1416/2844] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:41 2022] PUT /identity/v3/projects/b18bd6a37d984ac3b95eb58793679b7b/users/25a887c652214a8aa3113d1451dc09f7/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:41.744022 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ce9c2-9739-4069-b6e8-c9e72a5b71f0 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.744313 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ce9c2-9739-4069-b6e8-c9e72a5b71f0 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.744584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ce9c2-9739-4069-b6e8-c9e72a5b71f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.775059 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-c76ce9c2-9739-4069-b6e8-c9e72a5b71f0 None None] MFA Rules not processed for user `25a887c652214a8aa3113d1451dc09f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:41.776325 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c76ce9c2-9739-4069-b6e8-c9e72a5b71f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:41.851081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1429/2845] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:41.862624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:41.864663 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:41.898784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:41.931429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a2f6c-cb7b-47a7-be35-611ba891bb2b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:41.935620 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b92f22-d262-43e1-b4cd-ce64084e9a45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:41.936233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b92f22-d262-43e1-b4cd-ce64084e9a45 service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:41.936512 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b92f22-d262-43e1-b4cd-ce64084e9a45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:41.936779 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b92f22-d262-43e1-b4cd-ce64084e9a45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:41.946148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b92f22-d262-43e1-b4cd-ce64084e9a45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:41.947606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b92f22-d262-43e1-b4cd-ce64084e9a45 service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:41.957214 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1417/2846] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:42.363536 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ce9c2-9739-4069-b6e8-c9e72a5b71f0 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:42.429343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ce9c2-9739-4069-b6e8-c9e72a5b71f0 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:42.435823 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:42.436728 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:42.437177 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:42.437535 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] PATH_INFO: `/v3/users/58ca8d9f6084412b969c11e522c23733` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:42.439486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ca8d9f6084412b969c11e522c23733)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:42.442560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:42.496596 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ca8d9f6084412b969c11e522c23733'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:42.497334 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ca8d9f6084412b969c11e522c23733'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:42.503439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:42.505530 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ca8d9f6084412b969c11e522c23733'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:42.506294 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ca8d9f6084412b969c11e522c23733'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:42.514253 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ca8d9f6084412b969c11e522c23733'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:42.532106 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1430/2847] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:42 2022] DELETE /identity/v3/users/58ca8d9f6084412b969c11e522c23733 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:42.537190 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:42.537479 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:42.537751 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:42.557682 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:42.558885 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:42.619304 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1418/2848] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:42.629899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:42.631559 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:42.692624 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ac1df-24a0-4861-8c9c-5eed52b96e8e admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:42.696859 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:42.697468 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:42.697744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:42.698009 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:42.707880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:42.709318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:42.721576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1431/2849] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:43.012532 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:43.014075 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:43.027738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21036fc-060b-4515-bc30-09502baceb2e None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:43.033654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:43.035319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:43.035598 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:43.036016 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] PATH_INFO: `/v3/projects/5e95a012396b4e40ad7ee931586a5dc2` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:43.037121 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e95a012396b4e40ad7ee931586a5dc2)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:43.039128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:43.098635 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e95a012396b4e40ad7ee931586a5dc2', 'request_id': 'req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.107131 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e95a012396b4e40ad7ee931586a5dc2', 'request_id': 'req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.114782 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e95a012396b4e40ad7ee931586a5dc2', 'request_id': 'req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.129810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] The token cache is being invalidate because project 5e95a012396b4e40ad7ee931586a5dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:43.130099 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.130733 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.132309 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1419/2850] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:43 2022] DELETE /identity/v3/projects/5e95a012396b4e40ad7ee931586a5dc2 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:43.136438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:43.146625 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49596d-1a5c-48a2-b0d1-001843ad992e service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:43.210718 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:43.211265 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:43.211538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:43.211849 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] PATH_INFO: `/v3/users/25a887c652214a8aa3113d1451dc09f7` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:43.212964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25a887c652214a8aa3113d1451dc09f7)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:43.215283 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:43.263655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25a887c652214a8aa3113d1451dc09f7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.264277 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25a887c652214a8aa3113d1451dc09f7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.271269 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25a887c652214a8aa3113d1451dc09f7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.271889 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25a887c652214a8aa3113d1451dc09f7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.279788 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-3a2fee4a-73f1-45b4-8c55-c772a9d19204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25a887c652214a8aa3113d1451dc09f7'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.290020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:43.299081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1432/2851] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:54:43 2022] DELETE /identity/v3/users/25a887c652214a8aa3113d1451dc09f7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:43.590545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:43.592083 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:43.606378 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c15f3-1a08-43c0-bfc6-1fb2b334caf7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:43.665157 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:43.677810 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:43.678952 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:43.679229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:43.679523 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] PATH_INFO: `/v3/projects/b18bd6a37d984ac3b95eb58793679b7b` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:43.680801 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18bd6a37d984ac3b95eb58793679b7b)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:43.682795 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:43.738312 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18bd6a37d984ac3b95eb58793679b7b', 'request_id': 'req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.745412 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18bd6a37d984ac3b95eb58793679b7b', 'request_id': 'req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.753138 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18bd6a37d984ac3b95eb58793679b7b', 'request_id': 'req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.770202 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] The token cache is being invalidate because project b18bd6a37d984ac3b95eb58793679b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:54:43.770428 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.771106 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-f137bbb1-8b1c-40be-95ae-ae0e49f21d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:54:43.772929 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1420/2852] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:54:43 2022] DELETE /identity/v3/projects/b18bd6a37d984ac3b95eb58793679b7b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:43.778185 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7cdf9-d7ff-4583-bab4-194580a8c34e None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:43.778737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7cdf9-d7ff-4583-bab4-194580a8c34e None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:43.778737 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7cdf9-d7ff-4583-bab4-194580a8c34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:43.799337 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-0fa7cdf9-d7ff-4583-bab4-194580a8c34e None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:43.800623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0fa7cdf9-d7ff-4583-bab4-194580a8c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:43.859935 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1433/2853] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:43.868580 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba577e5-5f10-40d9-b0b2-3fbefaf26a45 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:43.868866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba577e5-5f10-40d9-b0b2-3fbefaf26a45 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:43.869142 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba577e5-5f10-40d9-b0b2-3fbefaf26a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:43.889299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-7ba577e5-5f10-40d9-b0b2-3fbefaf26a45 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:43.890507 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7ba577e5-5f10-40d9-b0b2-3fbefaf26a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:43.899868 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1421/2854] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:54:43.904320 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7cdf9-d7ff-4583-bab4-194580a8c34e None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:43.914607 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7cdf9-d7ff-4583-bab4-194580a8c34e None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:43.919550 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:43.920189 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:43.920466 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:43.920732 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:43.921378 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:54:43.921678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:43.923479 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:43.935362 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1434/2855] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:54:43.939991 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba577e5-5f10-40d9-b0b2-3fbefaf26a45 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:43.949515 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba577e5-5f10-40d9-b0b2-3fbefaf26a45 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:43.954319 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:43.954913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:43.955229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:43.955496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:43.956160 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:43.958044 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:43.988963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1422/2856] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:54:43 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:43.993057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:44.003193 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2433dc9-4699-4e40-aa48-e09561472aba admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:44.062086 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:44.062605 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:44.062884 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:44.063128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:44.063869 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:44.065797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:44.088798 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1435/2857] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:54:43 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:54:44.093528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:44.105058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3dc15-08c5-42d3-9caa-792c21d7bdaf admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:44.111015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:44.111676 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:44.111950 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:44.112213 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:44.112856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:44.114453 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:44.124654 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1423/2858] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 05:54:44.129074 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:44.140036 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b10d87-8e1b-4e3f-870c-b8831c4104a9 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:44.146609 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b7aa-9205-42dd-ad58-f802c5a36be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:44.147215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b7aa-9205-42dd-ad58-f802c5a36be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:44.147493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b7aa-9205-42dd-ad58-f802c5a36be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:44.147797 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b7aa-9205-42dd-ad58-f802c5a36be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:44.148446 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b7aa-9205-42dd-ad58-f802c5a36be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:44.150069 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b7aa-9205-42dd-ad58-f802c5a36be6 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:44.161641 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1436/2859] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:54:44.165797 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:44.176447 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65052f79-a1a7-435a-9501-6b720b9065d4 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:44.182267 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:44.182924 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:44.183204 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:44.183476 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] PATH_INFO: `/v3/projects/a50845ada6944e769765e36f978ca716/users/a821e4dcde67481bb33c574ac5973fd9/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:44.199721 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50845ada6944e769765e36f978ca716, user_id=a821e4dcde67481bb33c574ac5973fd9, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:44.202390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:44.217376 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1424/2860] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:54:44 2022] PUT /identity/v3/projects/a50845ada6944e769765e36f978ca716/users/a821e4dcde67481bb33c574ac5973fd9/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:54:44.223166 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e69b82-6cec-4f2a-895e-2059596555d5 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:44.223451 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e69b82-6cec-4f2a-895e-2059596555d5 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:44.223952 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e69b82-6cec-4f2a-895e-2059596555d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:44.244145 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-67e69b82-6cec-4f2a-895e-2059596555d5 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:44.245360 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-67e69b82-6cec-4f2a-895e-2059596555d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:44.297662 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:44.310143 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1437/2861] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:44.320562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:44.322554 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:44.388693 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e482d-f4cb-4ec2-965a-758e8bb4f91e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:44.393768 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:44.394405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:44.394671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:44.394940 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:44.405960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:44.408945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:44.417864 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1425/2862] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:45.056024 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c984a-1dd8-4edc-bc2b-56e55c7c3497 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:45.056893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c984a-1dd8-4edc-bc2b-56e55c7c3497 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:45.056893 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c984a-1dd8-4edc-bc2b-56e55c7c3497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:45.099178 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-fe8c984a-1dd8-4edc-bc2b-56e55c7c3497 None None] MFA Rules not processed for user `a821e4dcde67481bb33c574ac5973fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:45.101298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fe8c984a-1dd8-4edc-bc2b-56e55c7c3497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:45.184685 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1438/2863] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:45.198133 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:45.199633 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:45.226811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:54:45.261745 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0745c1-895c-4921-8f10-dc9e0dac385b service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:45.265623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:45.266234 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:45.266537 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:45.266826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:45.275277 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:45.276738 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:45.285623 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1426/2864] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:46.653689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:46.654248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:46.654248 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:46.673616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] MFA Rules not processed for user `a821e4dcde67481bb33c574ac5973fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:54:46.674842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:46.682448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1439/2865] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:54:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:54:46.929575 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:46.931244 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:46.944459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba017a-9a5a-472a-b47d-a7f6fd27e4eb service cinder] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:46.948695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:46.949348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:46.949621 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:46.949889 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:46.951619 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:46.961109 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:46.962528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:46.971965 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1427/2866] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:47.318132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:47.319649 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:47.395653 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e12a2-8e86-4a27-a4f9-2cae1ab83880 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:47.400556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:47.401163 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:47.401438 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:47.401706 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:47.403412 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:47.416382 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:47.418138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:47.429154 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1440/2867] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:55.900577 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:54:55.903116 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:55.981387 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b44214-598c-4b36-9335-afbba69fadfb service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:54:55.987025 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c9165-40eb-441e-b13c-0169a11e7b3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:54:55.987698 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c9165-40eb-441e-b13c-0169a11e7b3c service nova] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:54:55.987975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c9165-40eb-441e-b13c-0169a11e7b3c service nova] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:54:55.988276 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c9165-40eb-441e-b13c-0169a11e7b3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:54:55.989968 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-381c9165-40eb-441e-b13c-0169a11e7b3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:54:56.001710 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c9165-40eb-441e-b13c-0169a11e7b3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:54:56.003308 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c9165-40eb-441e-b13c-0169a11e7b3c service nova] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:54:56.013562 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1428/2868] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:54:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:54:57.423776 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=67 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:55:06.009375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=56 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:55:18.245208 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:55:18.246822 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:55:18.259445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf160db9-affa-4c29-8e4e-28dc56df7afc service placement] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:55:18.264800 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fcf79bef8d542d9adbfce89931032c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:55:18.265429 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:55:18.265703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:55:18.265965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:55:18.268298 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:55:18.331408 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:55:18.332880 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:55:18.343116 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1441/2869] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:55:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:55:28.337722 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=28 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:55:29.476199 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85218415-5b2f-4e2e-8d11-6717af1ef48c None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:55:29.477055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85218415-5b2f-4e2e-8d11-6717af1ef48c None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:55:29.477055 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85218415-5b2f-4e2e-8d11-6717af1ef48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:55:29.509000 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-85218415-5b2f-4e2e-8d11-6717af1ef48c None None] MFA Rules not processed for user `bb6d35f57b804ba5afa4ab86c46cd7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:55:29.510906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-85218415-5b2f-4e2e-8d11-6717af1ef48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:55:29.522181 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1429/2870] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:55:39.499874 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:55:55.413993 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:55:55.424818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1ff4b-09cb-442f-a9d9-90b0325ec899 service cinder] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:55:55.428770 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:55:55.429386 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:55:55.429645 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:55:55.429918 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:55:55.431940 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:55:55.500702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:55:55.502078 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:55:55.512067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1442/2871] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:55:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:56:05.507448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=30 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:56:08.799732 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3bfd0-2a83-4cfd-95ec-555e36689bf3 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:56:08.800282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3bfd0-2a83-4cfd-95ec-555e36689bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:56:08.800282 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3bfd0-2a83-4cfd-95ec-555e36689bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:56:08.821137 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-94e3bfd0-2a83-4cfd-95ec-555e36689bf3 None None] MFA Rules not processed for user `a821e4dcde67481bb33c574ac5973fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:56:08.822357 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-94e3bfd0-2a83-4cfd-95ec-555e36689bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:56:08.830960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1430/2872] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:56:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:56:11.823525 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:56:11.836577 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f38ed-49b5-4dc1-ae50-cc18ace3e812 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:56:11.844426 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:56:11.845213 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:56:11.845808 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:56:11.846077 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:56:11.848000 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:56:11.861310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:56:11.862902 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:56:11.874623 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1443/2873] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:56:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:56:18.815549 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:56:21.869903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=6 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:56:36.983350 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3bfd0-2a83-4cfd-95ec-555e36689bf3 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:56:37.052695 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3bfd0-2a83-4cfd-95ec-555e36689bf3 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:56:37.058757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:56:37.059389 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:56:37.059693 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:56:37.059955 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:56:37.062117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:56:37.105022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:56:37.106637 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:56:37.116890 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1431/2874] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:56:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:56:47.112511 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=36 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:57:14.171324 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:14.229531 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e027a8-8dc9-4b0d-9c5e-cba65027edab service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:14.234832 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:14.235493 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:14.235833 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:14.236105 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] PATH_INFO: `/v3/users/a821e4dcde67481bb33c574ac5973fd9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:14.237254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a821e4dcde67481bb33c574ac5973fd9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:14.239226 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:14.292842 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a821e4dcde67481bb33c574ac5973fd9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.293601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a821e4dcde67481bb33c574ac5973fd9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.301108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a821e4dcde67481bb33c574ac5973fd9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.301774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a821e4dcde67481bb33c574ac5973fd9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.310678 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6f141625-5225-414a-ab62-408bdd5dd931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a821e4dcde67481bb33c574ac5973fd9'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.328254 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1444/2875] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:57:14 2022] DELETE /identity/v3/users/a821e4dcde67481bb33c574ac5973fd9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:57:14.516399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:14.527709 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034cf44-a0c2-48cb-a2f4-ff1c1a64f107 service placement] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:14.593900 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:14.594541 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:14.594822 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:14.595095 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] PATH_INFO: `/v3/projects/a50845ada6944e769765e36f978ca716` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:14.596236 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50845ada6944e769765e36f978ca716)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:14.598158 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:14.652816 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50845ada6944e769765e36f978ca716', 'request_id': 'req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.658834 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50845ada6944e769765e36f978ca716', 'request_id': 'req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.665098 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50845ada6944e769765e36f978ca716', 'request_id': 'req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.693360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] The token cache is being invalidate because project a50845ada6944e769765e36f978ca716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:57:14.693568 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.694408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-c82aaa0f-010f-4b4d-b397-b9fafb533c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:57:14.696094 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1432/2876] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:57:14 2022] DELETE /identity/v3/projects/a50845ada6944e769765e36f978ca716 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:57:14.702570 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b59c5-294b-409f-bcda-30cf08ab6ce8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:14.702857 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b59c5-294b-409f-bcda-30cf08ab6ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:14.703131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b59c5-294b-409f-bcda-30cf08ab6ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:14.767560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-3c5b59c5-294b-409f-bcda-30cf08ab6ce8 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:14.768809 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-3c5b59c5-294b-409f-bcda-30cf08ab6ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:14.836711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1445/2877] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:57:14.845421 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0db676-b8d8-49e2-9c6a-e5269f9d3036 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:14.845826 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0db676-b8d8-49e2-9c6a-e5269f9d3036 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:14.846101 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0db676-b8d8-49e2-9c6a-e5269f9d3036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:14.867425 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-7b0db676-b8d8-49e2-9c6a-e5269f9d3036 None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:14.868696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-7b0db676-b8d8-49e2-9c6a-e5269f9d3036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:14.877505 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1433/2878] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:57:14.884403 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b59c5-294b-409f-bcda-30cf08ab6ce8 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:14.891268 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:57:14.897544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b59c5-294b-409f-bcda-30cf08ab6ce8 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:14.904715 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:14.905693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:14.906051 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:14.906551 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:14.907210 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81585) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 05:57:14.907522 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:14.909683 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:14.921762 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1446/2879] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 22 05:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 05:57:14.926415 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0db676-b8d8-49e2-9c6a-e5269f9d3036 None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:14.936353 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0db676-b8d8-49e2-9c6a-e5269f9d3036 None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:14.941490 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:14.942120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:14.942397 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:14.942667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:14.943280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:14.945299 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:14.974846 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1434/2880] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:57:14 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:57:14.980972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:14.992492 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e0f57-611e-4b9b-b931-2c86c762b0e1 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:15.053076 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:15.053689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:15.053961 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:15.054228 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:15.054850 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:15.056811 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:15.078839 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1447/2881] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:57:14 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:57:15.083402 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:15.094496 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2dd28-b678-43c4-aa82-d1061e75639d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:15.100784 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:15.101405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:15.101785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:15.102080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:15.102726 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:15.104377 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:15.115787 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1435/2882] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:57:15.119914 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:15.130384 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339400b7-9372-4ba3-86bd-ac818822b484 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:15.136175 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27035b67-56b2-4f6b-989c-5290d3e527f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:15.136773 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27035b67-56b2-4f6b-989c-5290d3e527f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:15.137054 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27035b67-56b2-4f6b-989c-5290d3e527f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:15.137318 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27035b67-56b2-4f6b-989c-5290d3e527f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:15.137964 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27035b67-56b2-4f6b-989c-5290d3e527f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:15.140173 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27035b67-56b2-4f6b-989c-5290d3e527f9 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:15.151700 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1448/2883] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:57:15.156594 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:15.171529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6108e-ad75-4c38-a48c-9cc0d98f7ab2 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:15.176871 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:15.177524 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:15.177808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:15.178079 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] PATH_INFO: `/v3/projects/5f84b1b8d6e44140b80a668e08afcaff/users/5e119e8e09aa4a8bb7cc96952f292833/roles/69140820db3a4dfea9428bdde1203a39` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:15.192808 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f84b1b8d6e44140b80a668e08afcaff, user_id=5e119e8e09aa4a8bb7cc96952f292833, role_id=69140820db3a4dfea9428bdde1203a39)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:15.195510 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:15.210412 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1436/2884] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:57:15 2022] PUT /identity/v3/projects/5f84b1b8d6e44140b80a668e08afcaff/users/5e119e8e09aa4a8bb7cc96952f292833/roles/69140820db3a4dfea9428bdde1203a39 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:57:15.215818 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0086d-2786-4c87-b9ee-3e551ac748ee None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:15.216311 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0086d-2786-4c87-b9ee-3e551ac748ee None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:15.216433 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0086d-2786-4c87-b9ee-3e551ac748ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:15.237292 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-61b0086d-2786-4c87-b9ee-3e551ac748ee None None] MFA Rules not processed for user `c35278cad6234455851a9ea018219e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:15.238498 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-61b0086d-2786-4c87-b9ee-3e551ac748ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:15.299763 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1449/2885] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:57:15.309626 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:15.311408 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:15.330727 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:57:15.392608 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59d0b7-32f0-4936-9240-f8cb2f7c57e7 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:15.396963 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:15.397590 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:15.397847 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:15.398117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:15.407911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:15.409347 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:15.419037 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1437/2886] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:57:19.481445 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d1af8-0903-42a5-85cd-9bb990423cb8 None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:19.482021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d1af8-0903-42a5-85cd-9bb990423cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:19.482021 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d1af8-0903-42a5-85cd-9bb990423cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:19.516469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-fb0d1af8-0903-42a5-85cd-9bb990423cb8 None None] MFA Rules not processed for user `5e119e8e09aa4a8bb7cc96952f292833`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:19.517703 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-fb0d1af8-0903-42a5-85cd-9bb990423cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:19.579357 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:57:19.586164 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1450/2887] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:57:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:57:19.590699 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:19.600692 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ff84a-f9c0-429b-98a0-f61d8c5e7212 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:19.605230 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:19.605845 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:19.606120 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:19.606390 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:19.607024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:19.609057 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:19.641292 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1438/2888] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 22 05:57:19 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 05:57:19.645413 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d1af8-0903-42a5-85cd-9bb990423cb8 None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:19.657632 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d1af8-0903-42a5-85cd-9bb990423cb8 None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:19.713774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:19.714774 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:19.715047 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:19.715448 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] PATH_INFO: `/v3/users` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:19.716014 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:19.718045 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:19.742867 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1451/2889] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 22 05:57:19 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 05:57:19.746839 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:19.758352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff92a7-a8ff-4c8c-8942-e8c25d296f9d admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:19.763264 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:19.764128 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:19.764438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:19.764555 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:19.765348 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:19.767053 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:19.777746 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1439/2890] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:57:19.782197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:19.792335 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2fb80-2fc9-46c6-823a-edefd2164e12 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:19.797637 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:19.798289 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:19.798560 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:19.798824 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] PATH_INFO: `/v3/projects/d224b30abdd243f581397ef684fb7858/users/3ff06acfae3e4d438864957210b4a66a/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:19.814556 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d224b30abdd243f581397ef684fb7858, user_id=3ff06acfae3e4d438864957210b4a66a, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:19.817237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:19.832067 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1452/2891] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:57:19 2022] PUT /identity/v3/projects/d224b30abdd243f581397ef684fb7858/users/3ff06acfae3e4d438864957210b4a66a/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:57:19.836022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:19.846945 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd232ad-2a6d-473e-8428-5ccb9dae2b9b admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:19.906543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5999d65-237a-4cea-bd89-7deea01bb6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:19.907151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5999d65-237a-4cea-bd89-7deea01bb6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:19.907422 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5999d65-237a-4cea-bd89-7deea01bb6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:19.907791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5999d65-237a-4cea-bd89-7deea01bb6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:19.908375 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5999d65-237a-4cea-bd89-7deea01bb6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:19.909996 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5999d65-237a-4cea-bd89-7deea01bb6e6 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:19.920195 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1440/2892] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 22 05:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 05:57:19.924862 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:19.936150 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b46db3-ff9b-4528-a16f-56f80612f50f admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:19.941310 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:19.942079 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:19.942439 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:19.942768 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] PATH_INFO: `/v3/projects/d224b30abdd243f581397ef684fb7858/users/3ff06acfae3e4d438864957210b4a66a/roles/6d5475aef2bb4ef98d9ff171fea3b73a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:19.944783 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d224b30abdd243f581397ef684fb7858, user_id=3ff06acfae3e4d438864957210b4a66a, role_id=6d5475aef2bb4ef98d9ff171fea3b73a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:19.947456 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:19.958660 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1453/2893] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 22 05:57:19 2022] PUT /identity/v3/projects/d224b30abdd243f581397ef684fb7858/users/3ff06acfae3e4d438864957210b4a66a/roles/6d5475aef2bb4ef98d9ff171fea3b73a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:57:24.524130 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d1ec-4b61-43c3-89dc-b8fe49ee36f5 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:24.524703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d1ec-4b61-43c3-89dc-b8fe49ee36f5 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:24.524703 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d1ec-4b61-43c3-89dc-b8fe49ee36f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:24.562911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-12d4d1ec-4b61-43c3-89dc-b8fe49ee36f5 None None] MFA Rules not processed for user `3ff06acfae3e4d438864957210b4a66a`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:24.564911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-12d4d1ec-4b61-43c3-89dc-b8fe49ee36f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:24.622272 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:57:24.644811 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1441/2894] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:57:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:57:24.658215 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:24.660159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:24.747618 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c1749-c8cf-49aa-a605-856db9e14ebc admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:24.751549 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69d9ee15b5fa43248fbdca69c3c91a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:24.752205 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:24.752483 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:24.752755 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:24.800160 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:57:24.816601 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:24.818073 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:24.828123 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1454/2895] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:57:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:57:24.953360 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c2918-c006-4633-84b2-31fb52e388b7 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:24.953666 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c2918-c006-4633-84b2-31fb52e388b7 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:24.953856 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c2918-c006-4633-84b2-31fb52e388b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:25.011706 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-4e2c2918-c006-4633-84b2-31fb52e388b7 None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:25.012906 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-4e2c2918-c006-4633-84b2-31fb52e388b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:25.081757 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1442/2896] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:57:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:57:25.142224 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:25.144576 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:25.245039 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c5dd1-9469-45c7-9318-ffcd746efe14 service glance] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:25.250554 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:25.251159 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:25.251435 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:25.251727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:25.260201 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:25.261785 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:25.269899 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1455/2897] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:57:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:57:34.670791 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70548482-656e-4024-a448-2fdc98efda62 None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:34.671354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70548482-656e-4024-a448-2fdc98efda62 None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:34.671354 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70548482-656e-4024-a448-2fdc98efda62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:34.693567 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-70548482-656e-4024-a448-2fdc98efda62 None None] MFA Rules not processed for user `5e119e8e09aa4a8bb7cc96952f292833`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:34.694771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-70548482-656e-4024-a448-2fdc98efda62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:34.702971 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1443/2898] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 22 05:57:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 05:57:34.975547 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:34.977135 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:35.037929 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467298b4-e572-4b55-8ee7-4da944cae797 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:35.041894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:35.042513 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:35.042793 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:35.043057 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:35.044894 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:35.056109 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:35.057544 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:35.066487 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1456/2899] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:57:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:57:37.974960 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664c458-e1fe-49ce-acc9-2cf54d9dedbc None None] REQUEST_METHOD: `POST` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:37.975543 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664c458-e1fe-49ce-acc9-2cf54d9dedbc None None] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:37.975835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664c458-e1fe-49ce-acc9-2cf54d9dedbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:38.008093 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.auth.core [None req-c664c458-e1fe-49ce-acc9-2cf54d9dedbc None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 05:57:38.009355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-c664c458-e1fe-49ce-acc9-2cf54d9dedbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:57:38.026080 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1444/2900] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 22 05:57:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 05:57:38.038779 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:57:38.052687 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b507e2b-720c-40c7-8d96-50440190cd18 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:57:38.058301 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:57:38.058985 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:57:38.059319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:57:38.059828 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:57:38.068469 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:57:38.070155 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:57:38.078286 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1457/2901] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 05:57:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 05:57:48.019624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=59 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:57:48.075108 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=80 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:58:08.055785 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664c458-e1fe-49ce-acc9-2cf54d9dedbc None None] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:08.066450 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664c458-e1fe-49ce-acc9-2cf54d9dedbc None None] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:08.070429 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:08.071024 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:08.071295 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:08.071559 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:08.081148 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:08.082592 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:08.093303 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1445/2902] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:58:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:58:11.533065 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:11.535584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:11.703600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1bcf-7a48-4911-85ac-4b4772ff2974 service swift] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:11.713090 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b919bcbb104fa3bf11847b6a35a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:11.714257 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:11.715432 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:11.715841 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:11.717861 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:11.739194 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:11.743745 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:11.760486 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1458/2903] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:58:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:58:14.400250 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:14.409278 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a13373-2a6f-4664-bc2c-72dfab3c0690 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:14.413086 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:14.414175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:14.414454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:14.414866 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:14.416680 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:14.432419 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:14.434174 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:14.446046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1446/2904] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:58:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:58:15.649980 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:15.660336 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc773c7d-9bb6-4060-bd1b-4f262f3d4102 service nova] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:15.664527 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:15.665142 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:15.665421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:15.665689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:15.667787 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:15.682138 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:15.683699 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:15.695206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1459/2905] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:58:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:58:24.303509 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:24.312646 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed52dc-3802-481e-8c0a-aeaf87210292 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:24.316529 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:24.317123 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:24.317399 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:24.317667 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:24.319528 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:24.390091 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:24.391545 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:24.402835 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1447/2906] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:58:25.690007 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=49 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:58:30.863711 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:30.865258 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:30.931891 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:30.933493 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:30.943095 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e75b8-9392-4eed-9bcf-e25d9f6e0720 service neutron] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:30.948015 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65615e8-7f68-447b-ac9e-69afc0088620 service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:30.950012 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:30.950652 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:30.951608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:30.951608 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:30.953066 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:30.953198 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:30.953771 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:30.954036 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:30.954313 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] PATH_INFO: `/v3/users/bb6d35f57b804ba5afa4ab86c46cd7c3` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:30.955438 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6d35f57b804ba5afa4ab86c46cd7c3)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:30.957409 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:30.965726 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:30.967305 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:30.979020 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1460/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:58:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:58:31.010897 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6d35f57b804ba5afa4ab86c46cd7c3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.011966 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6d35f57b804ba5afa4ab86c46cd7c3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.012671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6d35f57b804ba5afa4ab86c46cd7c3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.021283 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6d35f57b804ba5afa4ab86c46cd7c3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.028280 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6d35f57b804ba5afa4ab86c46cd7c3'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.048724 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1448/2908] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:58:30 2022] DELETE /identity/v3/users/bb6d35f57b804ba5afa4ab86c46cd7c3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:58:31.235430 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:31.254317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b486ca-f074-45ea-b994-65a3abd02fcb service placement] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:31.314689 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:31.315317 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:31.315584 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:31.315878 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] PATH_INFO: `/v3/projects/6d3a9f50ec2d4fff938fcd6a79f5cdc9` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:31.317148 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3a9f50ec2d4fff938fcd6a79f5cdc9)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:31.319114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:31.366197 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3a9f50ec2d4fff938fcd6a79f5cdc9', 'request_id': 'req-dbfc800b-8a77-4c3b-9519-681d3332bd9b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.371611 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3a9f50ec2d4fff938fcd6a79f5cdc9', 'request_id': 'req-dbfc800b-8a77-4c3b-9519-681d3332bd9b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.378114 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3a9f50ec2d4fff938fcd6a79f5cdc9', 'request_id': 'req-dbfc800b-8a77-4c3b-9519-681d3332bd9b'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.394663 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] The token cache is being invalidate because project 6d3a9f50ec2d4fff938fcd6a79f5cdc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:58:31.394949 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.395600 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.397128 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1461/2909] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:58:31 2022] DELETE /identity/v3/projects/6d3a9f50ec2d4fff938fcd6a79f5cdc9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:58:31.401514 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:31.402926 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:31.416273 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79302982-8e26-47cd-bab7-40f02e43fbba admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:31.455566 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:58:31.482370 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:31.483030 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:31.483310 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:31.483606 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] PATH_INFO: `/v3/users/44c09cbf366c4157a05471bef9491e61` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:31.484877 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c09cbf366c4157a05471bef9491e61)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:31.487189 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:31.539022 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c09cbf366c4157a05471bef9491e61'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.539730 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c09cbf366c4157a05471bef9491e61'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.540410 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c09cbf366c4157a05471bef9491e61'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.548783 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c09cbf366c4157a05471bef9491e61'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.555780 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c09cbf366c4157a05471bef9491e61'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.573557 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1449/2910] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 05:58:31 2022] DELETE /identity/v3/users/44c09cbf366c4157a05471bef9491e61 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 05:58:31.752927 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:58:31.754538 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:58:31.768693 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc800b-8a77-4c3b-9519-681d3332bd9b admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:58:31.787083 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:58:31.837582 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:58:31.838237 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:58:31.838515 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:58:31.838777 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] PATH_INFO: `/v3/projects/60f858018f734a1cba34dec09aec0f1a` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:58:31.840323 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f858018f734a1cba34dec09aec0f1a)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:58:31.842346 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:58:31.887704 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f858018f734a1cba34dec09aec0f1a', 'request_id': 'req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.893702 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f858018f734a1cba34dec09aec0f1a', 'request_id': 'req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.900319 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f858018f734a1cba34dec09aec0f1a', 'request_id': 'req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.915730 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] The token cache is being invalidate because project 60f858018f734a1cba34dec09aec0f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 05:58:31.916006 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.916727 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-24dfd761-cb7c-46ef-86a1-01ed301d4b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 05:58:31.918306 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1462/2911] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 05:58:31 2022] DELETE /identity/v3/projects/60f858018f734a1cba34dec09aec0f1a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 05:58:41.570781 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:58:41.912882 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 05:59:35.026054 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 05:59:35.028117 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:59:35.089352 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21e515-9b07-480b-8b15-9eebd1cb8ad3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 05:59:35.138270 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5c594aafc14c1e86ffa9702f9be964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 05:59:35.138872 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 05:59:35.139151 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 05:59:35.139420 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 05:59:35.141326 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 05:59:35.151084 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 05:59:35.152624 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 05:59:35.160664 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1450/2912] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 05:59:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 05:59:45.157233 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=46 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 06:00:09.849444 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] REQUEST_METHOD: `POST` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:00:09.849972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:00:09.849972 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:00:09.869925 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.auth.core [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] MFA Rules not processed for user `9de81bd996d4464d8a299228be695e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81585) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 06:00:09.871157 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:00:09.931251 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1463/2913] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 22 06:00:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 06:00:09.975848 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 06:00:09.977417 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:00:10.057475 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04becb24-78de-4228-a15f-82c56d3848ef service neutron] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 06:00:10.062343 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b63e2db3e07c4471be6928ff31334816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 06:00:10.062951 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:00:10.063229 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:00:10.063495 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:00:10.071430 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 06:00:10.073046 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 06:00:10.081042 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1451/2914] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 22 06:00:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 06:00:16.736655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 06:00:16.738717 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:00:16.804454 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 06:00:16.807220 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:00:16.830231 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0d01f-961b-4119-9415-3a9a2ae6ea5d None None] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 06:00:16.835714 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 06:00:16.836343 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] REQUEST_METHOD: `GET` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:00:16.836628 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:00:16.836903 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:00:16.839376 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:00:16.855712 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 06:00:16.857307 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 06:00:16.869295 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1464/2915] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 06:00:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 06:00:16.888799 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bdddb-7687-4863-a898-286cfbda903b service swift] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 06:00:16.893231 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 06:00:16.893913 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:00:16.894296 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:00:16.894774 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] PATH_INFO: `/v3/users/5e119e8e09aa4a8bb7cc96952f292833` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:00:16.896569 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e119e8e09aa4a8bb7cc96952f292833)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 06:00:16.898513 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 06:00:16.959449 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e119e8e09aa4a8bb7cc96952f292833'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:16.960161 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e119e8e09aa4a8bb7cc96952f292833'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:16.960740 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e119e8e09aa4a8bb7cc96952f292833'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:16.970293 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e119e8e09aa4a8bb7cc96952f292833'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:16.977391 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e119e8e09aa4a8bb7cc96952f292833'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:16.995671 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1452/2916] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 06:00:16 2022] DELETE /identity/v3/users/5e119e8e09aa4a8bb7cc96952f292833 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 06:00:17.182081 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 06:00:17.193912 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ad2b-4ec8-4d70-bd38-df50b80a44b1 service placement] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 06:00:17.250225 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 06:00:17.250858 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:00:17.251131 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:00:17.251401 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] PATH_INFO: `/v3/projects/5f84b1b8d6e44140b80a668e08afcaff` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:00:17.252534 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f84b1b8d6e44140b80a668e08afcaff)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 06:00:17.254647 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 06:00:17.293421 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 06:00:17.303181 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f84b1b8d6e44140b80a668e08afcaff', 'request_id': 'req-6a72ccca-c9a2-47d1-9509-4499f153ee92'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.308503 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f84b1b8d6e44140b80a668e08afcaff', 'request_id': 'req-6a72ccca-c9a2-47d1-9509-4499f153ee92'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.314495 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f84b1b8d6e44140b80a668e08afcaff', 'request_id': 'req-6a72ccca-c9a2-47d1-9509-4499f153ee92'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.328375 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] The token cache is being invalidate because project 5f84b1b8d6e44140b80a668e08afcaff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 06:00:17.328655 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.329261 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.330744 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1465/2917] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 06:00:17 2022] DELETE /identity/v3/projects/5f84b1b8d6e44140b80a668e08afcaff => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 06:00:17.334970 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 06:00:17.336455 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:00:17.350574 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95677a33-bf92-4e34-bbef-3b7dbdbcd36e admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 06:00:17.355975 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 06:00:17.410776 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 06:00:17.411405 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:00:17.411684 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:00:17.411989 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] PATH_INFO: `/v3/users/3ff06acfae3e4d438864957210b4a66a` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:00:17.413152 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff06acfae3e4d438864957210b4a66a)` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 06:00:17.415063 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 06:00:17.462979 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff06acfae3e4d438864957210b4a66a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.463911 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff06acfae3e4d438864957210b4a66a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.464499 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff06acfae3e4d438864957210b4a66a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.473175 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff06acfae3e4d438864957210b4a66a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.480355 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.notifications [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff06acfae3e4d438864957210b4a66a'} {{(pid=81584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.497645 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1453/2918] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 22 06:00:17 2022] DELETE /identity/v3/users/3ff06acfae3e4d438864957210b4a66a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 06:00:17.688965 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Authenticating user token {{(pid=81585) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 06:00:17.690610 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.fernet_utils [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81585) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:00:17.704616 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72ccca-c9a2-47d1-9509-4499f153ee92 admin admin] Validating token access rules against request {{(pid=81585) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 06:00:17.766206 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c35278cad6234455851a9ea018219e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb2b2de175ff44f7b8683cb1604be1f7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81585) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 06:00:17.766897 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:00:17.767098 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:00:17.767383 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] PATH_INFO: `/v3/projects/d224b30abdd243f581397ef684fb7858` {{(pid=81585) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:00:17.768654 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d224b30abdd243f581397ef684fb7858)` {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 06:00:17.770591 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] RBAC: Authorization granted {{(pid=81585) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 06:00:17.831954 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd224b30abdd243f581397ef684fb7858', 'request_id': 'req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.837457 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd224b30abdd243f581397ef684fb7858', 'request_id': 'req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.843922 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd224b30abdd243f581397ef684fb7858', 'request_id': 'req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d'} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.858214 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] The token cache is being invalidate because project d224b30abdd243f581397ef684fb7858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81585) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 06:00:17.858500 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.859132 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG keystone.notifications [None req-17c7e77e-5f2a-4e3d-8e23-61636cca3b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81585) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 06:00:17.860651 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: [pid: 81585|app: 0|req: 1466/2919] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 22 06:00:17 2022] DELETE /identity/v3/projects/d224b30abdd243f581397ef684fb7858 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 06:00:27.494314 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 06:00:27.855742 dv-cinder-853910-1-iscsi devstack@keystone.service[81585]: DEBUG dbcounter [-] [81585] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81585) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 06:01:42.263899 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Authenticating user token {{(pid=81584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 06:01:42.265486 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:01:42.341725 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a80660-6547-4cb4-a926-ea474f2ecbc3 admin admin] Validating token access rules against request {{(pid=81584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 06:01:42.409852 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abf1f0-592e-4bb8-9e9f-7d329f17316b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '733035fd9ee6414dbc23c69bb9206b6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62d5a029139c4245a36acda923371996', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 06:01:42.410459 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf1f0-592e-4bb8-9e9f-7d329f17316b service placement] REQUEST_METHOD: `GET` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 06:01:42.410737 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf1f0-592e-4bb8-9e9f-7d329f17316b service placement] SCRIPT_NAME: `/identity` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 06:01:42.410998 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf1f0-592e-4bb8-9e9f-7d329f17316b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 06:01:42.413058 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.fernet_utils [None req-e5abf1f0-592e-4bb8-9e9f-7d329f17316b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 06:01:42.425065 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abf1f0-592e-4bb8-9e9f-7d329f17316b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 06:01:42.426550 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abf1f0-592e-4bb8-9e9f-7d329f17316b service placement] RBAC: Authorization granted {{(pid=81584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 06:01:42.436696 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: [pid: 81584|app: 0|req: 1454/2920] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 22 06:01:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 06:01:52.432380 dv-cinder-853910-1-iscsi devstack@keystone.service[81584]: DEBUG dbcounter [-] [81584] Writing DB stats keystone:SELECT=58 {{(pid=81584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}